-- Logs begin at Wed 2022-09-07 12:21:02 UTC, end at Wed 2022-09-07 13:45:28 UTC. -- Sep 07 12:58:26.816704 dv-cinder-804035-50-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 07 12:58:26.832663 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 7 12:58:26 2022] *** Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: nodename: dv-cinder-804035-50-iscsi Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: machine: x86_64 Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: clock source: unix Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: pcre jit disabled Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: detected number of CPU cores: 4 Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: current working directory: / Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: detected binary path: /usr/bin/uwsgi-core Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: your processes number limit is 47680 Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: your memory page size is 4096 bytes Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: detected max file descriptor number: 2048 Sep 07 12:58:26.835966 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: lock engine: pthread robust mutexes Sep 07 12:58:26.837325 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: thunder lock: enabled Sep 07 12:58:26.837325 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 07 12:58:26.837325 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 07 12:58:26.908648 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: Python main interpreter initialized at 0x55e475adb9c0 Sep 07 12:58:26.908648 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: python threads support enabled Sep 07 12:58:26.908648 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: your server socket listen backlog is limited to 100 connections Sep 07 12:58:26.908648 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: your mercy for graceful operations on workers is 90 seconds Sep 07 12:58:26.909200 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: mapped 403077 bytes (393 KB) for 2 cores Sep 07 12:58:26.909359 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: *** Operational MODE: preforking *** Sep 07 12:58:26.909852 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: *** uWSGI is running in multiple interpreter mode *** Sep 07 12:58:26.909852 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: spawned uWSGI master process (pid: 81784) Sep 07 12:58:26.910148 dv-cinder-804035-50-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 07 12:58:26.915135 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: spawned uWSGI worker 1 (pid: 81795, cores: 1) Sep 07 12:58:26.915135 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: spawned uWSGI worker 2 (pid: 81796, cores: 1) Sep 07 12:58:26.915135 dv-cinder-804035-50-iscsi devstack@keystone.service[81784]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 07 12:58:31.815086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 12:58:31.815086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: from cryptography.utils import int_from_bytes Sep 07 12:58:31.816331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 12:58:31.816331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: from cryptography.utils import int_from_bytes Sep 07 12:58:31.869853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 12:58:31.870052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 12:58:31.870229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] command line args: [] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 12:58:31.870524 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 12:58:31.870728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ================================================================================ {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 12:58:31.871020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] admin_token = **** {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.871315 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] config_dir = [] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.871522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.871729 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] config_source = [] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.871909 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] debug = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.872109 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.873263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.873263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.873263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.873263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.873608 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] list_limit = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.873817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] log_config_append = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.874005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.874188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] log_dir = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.874367 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] log_file = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.875174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.875465 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.875710 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.875961 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.876118 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.876304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.876550 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.876817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.877057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.877292 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.877530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.877765 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.878012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.878261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.878542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.879369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.879529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] publish_errors = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.879771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.880004 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.880201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.880473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.880666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.880886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] standard_threads = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.881130 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.881365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.881521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.881701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] use_journal = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.881893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] use_json = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.882158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] use_stderr = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.882359 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] use_syslog = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.882599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:31.882887 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.883158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.883437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.883683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.883931 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.884179 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.884425 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.884664 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.884963 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.885191 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.885414 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.885650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.885899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.886135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.886381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.886630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.886868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.887176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.887406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.887654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.887891 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.888148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.888413 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.888674 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.888938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.889187 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.889441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.889675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.889914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.890163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.890468 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.891246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.891542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.891799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.892104 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.892389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.892645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.892883 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.893127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.893416 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.893651 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.893885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.894119 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.894355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.894596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.894854 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.895095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.895374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.895608 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.895847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.896083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.896332 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.896563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.896802 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.897043 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.897280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.897527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.897765 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.897997 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.898248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.898482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.898724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.898957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.899261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.899504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.899738 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.899978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.900278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.900555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.900796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.901046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.901335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.901578 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.901819 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.902055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.902305 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.902542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.902786 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.903026 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.903292 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.903540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.903782 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.904022 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.904257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.904501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.904774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.905051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.905295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.905548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.905805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.906087 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.906333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.906595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.906831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.907068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.907334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.907566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.907798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.908035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.908266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.908510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.908740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.908994 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.909235 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.909491 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.909757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.909997 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.910229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] auth.external = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.910462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.910766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.911001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.911261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] auth.password = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.911495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] auth.token = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.911736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.911977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.912223 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.912452 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.912708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.912962 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.913196 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.913434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] credential.caching = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.913683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.913939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.914189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.914446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.914693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.914930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.915391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.915521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.915701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.916079 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.916449 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.916709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.917023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.917345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.917683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.917928 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] federation.caching = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.918211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.918457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.918713 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.918944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.919237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.919504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.919723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.920051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.920333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.920659 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.920912 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.921155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.caching = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.921404 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.921718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.921957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.922194 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.922444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.922677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.922916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.923194 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.923451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.923684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.923918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.924151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.924399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.924643 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.924892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.925155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.925417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.925690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.925933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.926172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.926407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.926644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.926878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.927233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.927391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.927624 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.927873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.928106 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.928378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.928623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.928917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.929170 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.929418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.929648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.929892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.930119 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.930361 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.930598 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.930868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.931146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.931391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.931808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.932060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.932298 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.932558 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.932833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.933087 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.933329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.933583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.933843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.934081 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.934343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.934608 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.934844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.935163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.935373 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.935624 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.935861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.936149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.936396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.936634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.936879 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.937112 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.937883 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.938134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.938381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.938628 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.938872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.939130 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.939382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.939623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.939862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.940102 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.940339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.940583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.940831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.941072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.941327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.941559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.941807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.942045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.942284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.942526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.942826 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.943073 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.943350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.943682 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.943935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] resource.caching = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.944193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.944442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.944683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.944933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.945192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.945429 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.945699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.945939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.946227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.946465 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] role.caching = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.946712 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] role.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.946983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.947270 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.947529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.947780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.948029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.948297 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.948546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.948794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.949045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.949279 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.949527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.949773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.950020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.950268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.950731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.950966 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.951247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.951499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.951793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.952015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.952299 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.952560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.952849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.953617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.953872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.954107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.954356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.954612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.954936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.955296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.955458 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.955655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.955911 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.956151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] token.caching = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.956411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.956684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.956964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.957229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.957476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.957774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.957961 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.958221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.958471 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.958713 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.958954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.959221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.959472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.959758 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.959991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.960235 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.960507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.960764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.connection = **** {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.961010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.961265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.961497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.961746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.961982 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.962220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.962457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.962693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.962938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.963204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.963450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.963769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.963978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.964338 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.964702 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.964997 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.965244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.965485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:31.965704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81796) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 12:58:32.000645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.000910 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.001086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.012139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.044444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.044696 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.106352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.163140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.163388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.238578 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.238833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.238991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.239255 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.239481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.239705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.305653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 12:58:32.305653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: from cryptography.utils import int_from_bytes Sep 07 12:58:32.306553 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 12:58:32.306553 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: from cryptography.utils import int_from_bytes Sep 07 12:58:32.364237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.364479 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.364479 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.364866 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.364999 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.365155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.365384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.371803 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 12:58:32.372052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 12:58:32.372194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] command line args: [] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 12:58:32.372444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 12:58:32.372600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ================================================================================ {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 12:58:32.372868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] admin_token = **** {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.373119 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] config_dir = [] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.373304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.373487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] config_source = [] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.373709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] debug = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.373853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.374102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.374284 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.374695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.375412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.375618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] list_limit = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.375795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] log_config_append = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.376003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.376137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] log_dir = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.376296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] log_file = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.376534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.376783 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.377031 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.377279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.377453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.377627 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.377870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.378114 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.378340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.378480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.378613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.378791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.379323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.379574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.379819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.380115 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.380969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.381216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] publish_errors = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.381534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.381926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.382238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.382679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.382993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.383304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] standard_threads = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.383584 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.383985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.384261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.384490 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] use_journal = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.384712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] use_json = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.384961 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] use_stderr = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.385260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] use_syslog = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.385575 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 12:58:32.385892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.386169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.386443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.386692 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.386945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.387277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.387627 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.387909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.388194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.388449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.388707 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.389013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.389451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.389858 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.390253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.390605 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.390956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.391347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.391760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.392077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.392355 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.392552 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.392758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.392954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.392954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.393341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.393538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.393734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.393958 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.394097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.394246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.394246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81796) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.394486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.394486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.394486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.394870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.395277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.395631 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.395923 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.396194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.396452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.396708 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.396955 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.397209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.397516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.397822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.398153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.398559 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.398914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.399348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.399655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.399922 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.400184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.400423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.400669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.400885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.401133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.401375 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.401627 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.401860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.402101 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.402345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.402583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.402822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.403060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.403327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.403579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.403822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.404157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.404549 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.405016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.405335 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.405622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.405947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.406196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.406479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.406728 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.406982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.407406 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.407599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.407793 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.408035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.408366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.408712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.409108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.409499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.409928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.410335 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.410679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.411033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.411294 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.411492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.411698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.411897 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 12:58:32.412090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.412302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.412509 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.412708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 12:58:32.412893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.412893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.413258 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.413467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.413663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.413846 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.414024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.414219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.414393 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.414590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.414590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.414590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.414590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.414590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.415295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.415295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.415549 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.415782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.416029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.416381 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.416582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.416823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.417162 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.417438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.417801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] auth.external = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.418069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.418376 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.418610 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.418848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] auth.password = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.419083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] auth.token = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.419349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.419710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.420098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.420431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.420726 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.420988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.421227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.421994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] credential.caching = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.422413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.422829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.423288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.423642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.423935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.424075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.424273 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.424428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.424563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.424693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.424822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.424947 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.425102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.425236 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.425446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.425704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.425971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.426217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.426476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.426717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] federation.caching = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.426956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.427238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.427490 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.427785 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.428200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.428511 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.428855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.429231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.429503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.429750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.430007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.430251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.caching = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.430503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.430755 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.430995 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.431264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.431514 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.431761 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.432164 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.432359 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.432586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.433037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.433307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.433557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.433794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.434130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.434503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.434878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.435209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.435414 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.435553 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.435718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.435854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.436006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.436125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.436248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.436379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.436568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.436697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.436879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.437165 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.437811 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.437939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.438071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.438198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.438341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.438469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.438594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.438712 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.438836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.438970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.439129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.439258 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.439480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.439719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.439967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.440215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.440447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.440688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.441026 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.441332 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.441650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.442011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.442379 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.442667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.443090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.443435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.443722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.444098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.444532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.444936 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.445295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.445682 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.445983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.446391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.446734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.447141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.447457 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.447831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.448176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.448536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.448847 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.449194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.449566 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.449982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.450408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.450691 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.450959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.451240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.451496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.451852 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.452240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.452509 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.452821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.453174 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.453500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.453880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.454295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.454585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.455136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.455419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.455686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.455926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.456179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.456499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.456754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.457036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.457289 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.457537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.457773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.458015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.458397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] resource.caching = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.458776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.459197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.459612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.459943 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.460228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.460479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.460733 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.460978 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.461227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.461558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] role.caching = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.461991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] role.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.462372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.462473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.462742 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.463213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.463478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.463841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.464052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.464198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.464342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.464455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.464712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.465000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.465394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.465831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.466227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.466603 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.466916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.467488 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.467840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.468217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.469136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.469460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.469757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.470025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.470144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.470363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.470471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.470678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.470787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.471006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.471276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.471519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.471756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.472087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.472504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.472799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.473046 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.473293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.473531 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.473869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] token.caching = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.474261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.474514 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.474810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.475477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.475907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.476332 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.476774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.477129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.477505 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.477629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.477787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.477927 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.478050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.478272 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.478526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.478816 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.479056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.479338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.479612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.479883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.connection = **** {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.480171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.480563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.480939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.481242 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.481542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.481927 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.482245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.482496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.482622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.482751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.482885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.483011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.483165 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.483291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.483412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.483668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.483819 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.484096 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.484725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.484834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.484942 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.485088 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.485213 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.485422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.486005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.486352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 12:58:32.486602 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81795) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 12:58:32.501731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.502047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.506510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.509985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.510339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.515027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.518746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.522472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.522789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.523098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.523346 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.523484 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.523706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.523832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.524007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.524158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.541232 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.542821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.543209 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.543645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.543752 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.551438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.553050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.554620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.554907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.558417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.558831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.559395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.559694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.560054 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.572759 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.573131 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.573563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.573867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.574171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.574454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.574750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.579614 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.580023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.590905 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.594648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.597933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.599515 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.601508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.601918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.602213 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.609956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.611562 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.614854 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.615070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.615385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.615660 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.630535 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.630972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.631193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.638474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81796) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:32.638765 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.639062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.639373 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.639728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.640021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.640329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.640607 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.640936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.641219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.641510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.641784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.642046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.642311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81796) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:32.654723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.678131 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e475adb9c0 pid: 81796 (default app) Sep 07 12:58:32.681872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005e7cf-b3f9-4e80-a659-3172f5a6e7b5 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:32.682179 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005e7cf-b3f9-4e80-a659-3172f5a6e7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:32.682432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005e7cf-b3f9-4e80-a659-3172f5a6e7b5 None None] PATH_INFO: `/v3` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:32.686860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1/1] 10.222.0.61 () {50 vars in 791 bytes} [Wed Sep 7 12:58:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:58:32.724110 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.724495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.951914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.952192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.952528 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.952914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.953262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.953607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.977725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.978053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.978376 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.978710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.979091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.979460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.979784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:32.996503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:33.002964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:33.003398 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:33.003803 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:33.004440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:33.004833 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:33.005223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:33.005617 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81795) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 12:58:33.024732 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.025095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 12:58:33.025702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.026050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 12:58:33.026530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.026994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.027488 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.027960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.046080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.047080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.047572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.048068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.063142 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.068590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.069112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.069606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.070196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.070720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.071437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.071944 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.072449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.072993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.073433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.073948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.120798 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.123855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.124407 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.134989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.135542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.136028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.149589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.150125 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.158278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.158801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.159326 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.159850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.160338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.163914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.164422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.164922 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.197951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.198436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.206326 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.212820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.213349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.222406 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.228583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.235645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.236205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.236681 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.237459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.237967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.238464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.270879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.271417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.271859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.272287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.286448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.290066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.293145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.293647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.299585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.300091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.300601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.301096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.301589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.325125 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.325779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.326109 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.326591 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.327076 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.327606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.328651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.358414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.364687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.370682 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.373400 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.377379 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.377903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.378413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.392688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.395687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.401871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.402432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.402931 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.403459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.420069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.420598 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.421089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.435086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81795) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 12:58:33.435639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.436125 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.436610 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.437604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.438090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.438586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.439084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.439599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.440097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.440615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.441112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.441606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.442101 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81795) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 12:58:33.514219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55e475adb9c0 pid: 81795 (default app) Sep 07 12:58:40.105658 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5eaa3e-eae3-4a57-9bfb-f0eb2001fccb None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:40.106240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5eaa3e-eae3-4a57-9bfb-f0eb2001fccb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:40.106240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5eaa3e-eae3-4a57-9bfb-f0eb2001fccb None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:40.107730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 2/2] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:58:40.117148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba7fff-785a-433b-ad47-48c567728ccd None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:40.117907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba7fff-785a-433b-ad47-48c567728ccd None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:40.117907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba7fff-785a-433b-ad47-48c567728ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:40.209182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: INFO dbcounter [None req-1cba7fff-785a-433b-ad47-48c567728ccd None None] Registered counter for database keystone Sep 07 12:58:40.231710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG oslo_db.sqlalchemy.engines [None req-1cba7fff-785a-433b-ad47-48c567728ccd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81795) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 12:58:40.238752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writer thread running {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 12:58:40.444597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG passlib.handlers.bcrypt [None req-1cba7fff-785a-433b-ad47-48c567728ccd None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81795) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 12:58:40.444971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG passlib.handlers.bcrypt [None req-1cba7fff-785a-433b-ad47-48c567728ccd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81795) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 12:58:40.506879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-1cba7fff-785a-433b-ad47-48c567728ccd None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:40.508739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1cba7fff-785a-433b-ad47-48c567728ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:40.679307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1/3] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 566 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 07 12:58:41.178552 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5eaa3e-eae3-4a57-9bfb-f0eb2001fccb None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:41.267316 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: INFO dbcounter [None req-ad5eaa3e-eae3-4a57-9bfb-f0eb2001fccb None None] Registered counter for database keystone Sep 07 12:58:41.294104 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG oslo_db.sqlalchemy.engines [None req-ad5eaa3e-eae3-4a57-9bfb-f0eb2001fccb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81796) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 12:58:41.302823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writer thread running {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 12:58:41.456046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5eaa3e-eae3-4a57-9bfb-f0eb2001fccb None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:41.461804 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8f08b-97c0-40a2-b3e8-d2a63e1cb7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:41.462655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8f08b-97c0-40a2-b3e8-d2a63e1cb7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:41.462940 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8f08b-97c0-40a2-b3e8-d2a63e1cb7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:41.463273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8f08b-97c0-40a2-b3e8-d2a63e1cb7e2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:41.476816 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8f08b-97c0-40a2-b3e8-d2a63e1cb7e2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:41.548109 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: WARNING keystone.common.rbac_enforcer.enforcer [None req-48f8f08b-97c0-40a2-b3e8-d2a63e1cb7e2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 12:58:41.548594 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8f08b-97c0-40a2-b3e8-d2a63e1cb7e2 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:41.562529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 3/4] 10.222.0.61 () {58 vars in 1324 bytes} [Wed Sep 7 12:58:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 12:58:41.570433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba7fff-785a-433b-ad47-48c567728ccd None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:41.587022 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba7fff-785a-433b-ad47-48c567728ccd None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:41.592003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee77ef-11c9-4ae4-8800-a98bbae8b395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:41.593760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee77ef-11c9-4ae4-8800-a98bbae8b395 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:41.594045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee77ef-11c9-4ae4-8800-a98bbae8b395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:41.595030 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee77ef-11c9-4ae4-8800-a98bbae8b395 admin admin] PATH_INFO: `/v3/projects/2f256a069ba947c6a6b8cccfea6c7a22` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:41.596878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee77ef-11c9-4ae4-8800-a98bbae8b395 admin admin] RBAC: Authorizing `identity:get_project(project_id=2f256a069ba947c6a6b8cccfea6c7a22)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:41.653920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0cee77ef-11c9-4ae4-8800-a98bbae8b395 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 12:58:41.654318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee77ef-11c9-4ae4-8800-a98bbae8b395 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:41.660287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 2/5] 10.222.0.61 () {56 vars in 1207 bytes} [Wed Sep 7 12:58:41 2022] GET /identity/v3/projects/2f256a069ba947c6a6b8cccfea6c7a22 => generated 344 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:58:44.051632 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706ec31-db10-4d36-a13a-4fbbc6024389 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:44.052222 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706ec31-db10-4d36-a13a-4fbbc6024389 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:44.052222 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706ec31-db10-4d36-a13a-4fbbc6024389 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:44.056393 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 4/6] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:58:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:44.062736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8566f10-1c64-4915-8b66-ba8298846309 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:44.063301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8566f10-1c64-4915-8b66-ba8298846309 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:44.063458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8566f10-1c64-4915-8b66-ba8298846309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:44.087394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-b8566f10-1c64-4915-8b66-ba8298846309 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:44.089070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b8566f10-1c64-4915-8b66-ba8298846309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:44.097920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 3/7] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 07 12:58:44.376409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706ec31-db10-4d36-a13a-4fbbc6024389 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:44.390367 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706ec31-db10-4d36-a13a-4fbbc6024389 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:44.394975 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec66953-0ad1-4cea-a46a-91e0e7a90f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:44.395640 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec66953-0ad1-4cea-a46a-91e0e7a90f71 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:44.395926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec66953-0ad1-4cea-a46a-91e0e7a90f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:44.396201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec66953-0ad1-4cea-a46a-91e0e7a90f71 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:44.406174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec66953-0ad1-4cea-a46a-91e0e7a90f71 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:44.408049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec66953-0ad1-4cea-a46a-91e0e7a90f71 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:44.418113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 5/8] 10.222.0.61 () {58 vars in 1324 bytes} [Wed Sep 7 12:58:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 12:58:44.424210 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8566f10-1c64-4915-8b66-ba8298846309 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:44.434682 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8566f10-1c64-4915-8b66-ba8298846309 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:44.441836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab68b6-1cbe-456a-b246-e009e48434aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:44.442517 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab68b6-1cbe-456a-b246-e009e48434aa admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:44.442799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab68b6-1cbe-456a-b246-e009e48434aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:44.443073 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab68b6-1cbe-456a-b246-e009e48434aa admin admin] PATH_INFO: `/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:44.444648 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab68b6-1cbe-456a-b246-e009e48434aa admin admin] RBAC: Authorizing `identity:get_user(user_id=8bfb2d4384094e9e8a8b910061e5c9d4)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:44.446669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab68b6-1cbe-456a-b246-e009e48434aa admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:44.448413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 4/9] 10.222.0.61 () {56 vars in 1198 bytes} [Wed Sep 7 12:58:44 2022] GET /identity/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4 => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:58:46.848896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57145f2-62d5-43ff-ba71-eec2ee970232 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:46.849482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57145f2-62d5-43ff-ba71-eec2ee970232 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:46.849871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57145f2-62d5-43ff-ba71-eec2ee970232 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:46.852557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 6/10] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:58:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:46.861429 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c7b42-41e1-4584-a43b-7790f54ae9aa None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:46.862224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c7b42-41e1-4584-a43b-7790f54ae9aa None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:46.862536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c7b42-41e1-4584-a43b-7790f54ae9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:46.883825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8b6c7b42-41e1-4584-a43b-7790f54ae9aa None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:46.885246 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8b6c7b42-41e1-4584-a43b-7790f54ae9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:47.014095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 5/11] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:58:47.056382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d38ed-c03d-4dea-bc2b-3cb97ba9984b None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:47.056869 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d38ed-c03d-4dea-bc2b-3cb97ba9984b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:47.057366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d38ed-c03d-4dea-bc2b-3cb97ba9984b None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:47.059982 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 7/12] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:58:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:47.068664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f43880-9678-455a-a5d0-ce10172a7cbf None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:47.069317 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f43880-9678-455a-a5d0-ce10172a7cbf None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:47.069990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f43880-9678-455a-a5d0-ce10172a7cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:47.095767 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-17f43880-9678-455a-a5d0-ce10172a7cbf None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:47.097163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-17f43880-9678-455a-a5d0-ce10172a7cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:47.123196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:47.195141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 6/13] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:58:47.302897 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d38ed-c03d-4dea-bc2b-3cb97ba9984b None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:47.485735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f43880-9678-455a-a5d0-ce10172a7cbf None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:47.554396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d38ed-c03d-4dea-bc2b-3cb97ba9984b None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:47.597530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e101b-a662-4691-998f-8bd2d9af9a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:47.598540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e101b-a662-4691-998f-8bd2d9af9a63 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:47.598910 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e101b-a662-4691-998f-8bd2d9af9a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:47.599363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e101b-a662-4691-998f-8bd2d9af9a63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:47.616380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e101b-a662-4691-998f-8bd2d9af9a63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-dd7e101b-a662-4691-998f-8bd2d9af9a63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:58:47.626711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:58:47.628648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:58:47.630489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:58:47.630489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:58:47.630489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:58:47.630489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:58:47.630489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:58:47.630489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:58:47.630489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:58:47.630489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:58:47.630489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:58:47.630489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:58:47.630489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:58:47.630489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:58:47.630489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 8/14] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 12:58:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:58:47.638067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e101b-a662-4691-998f-8bd2d9af9a63 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:47.642927 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f43880-9678-455a-a5d0-ce10172a7cbf None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:47.686996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:47.691854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9d0b1-16de-4376-86b0-26b0f5e04768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:47.692586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9d0b1-16de-4376-86b0-26b0f5e04768 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:47.692970 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9d0b1-16de-4376-86b0-26b0f5e04768 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:47.693429 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9d0b1-16de-4376-86b0-26b0f5e04768 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:47.724057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:47.783485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e101b-a662-4691-998f-8bd2d9af9a63 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:47.789007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9d0b1-16de-4376-86b0-26b0f5e04768 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:47.790784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9d0b1-16de-4376-86b0-26b0f5e04768 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:47.834435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cd858-0509-4d80-b136-7ae5a0f37b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:47.835533 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cd858-0509-4d80-b136-7ae5a0f37b36 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:47.836029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cd858-0509-4d80-b136-7ae5a0f37b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:47.836528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cd858-0509-4d80-b136-7ae5a0f37b36 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:47.837785 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cd858-0509-4d80-b136-7ae5a0f37b36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:58:47.838345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cd858-0509-4d80-b136-7ae5a0f37b36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:47.841109 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cd858-0509-4d80-b136-7ae5a0f37b36 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:47.864940 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 9/15] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:58:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:58:47.874356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cd858-0509-4d80-b136-7ae5a0f37b36 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:47.910680 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 7/16] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:58:47.917568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9d0b1-16de-4376-86b0-26b0f5e04768 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:48.001527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:48.009662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cd858-0509-4d80-b136-7ae5a0f37b36 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:48.010447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:48.051062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799e40a-3373-4fa3-b6ad-086269da3106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:48.051798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799e40a-3373-4fa3-b6ad-086269da3106 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:48.052086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799e40a-3373-4fa3-b6ad-086269da3106 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:48.052360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799e40a-3373-4fa3-b6ad-086269da3106 None admin] PATH_INFO: `/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:48.054085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799e40a-3373-4fa3-b6ad-086269da3106 None admin] RBAC: Authorizing `identity:get_user(user_id=8bfb2d4384094e9e8a8b910061e5c9d4)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:48.055695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799e40a-3373-4fa3-b6ad-086269da3106 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:48.057393 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 10/17] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:58:47 2022] GET /identity/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:58:48.063239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799e40a-3373-4fa3-b6ad-086269da3106 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:48.106482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9d0b1-16de-4376-86b0-26b0f5e04768 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:48.160766 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e1ad4-4d93-4291-ad33-468e2e4858d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:48.161571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e1ad4-4d93-4291-ad33-468e2e4858d7 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:48.161912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e1ad4-4d93-4291-ad33-468e2e4858d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:48.162281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e1ad4-4d93-4291-ad33-468e2e4858d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:48.175863 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e1ad4-4d93-4291-ad33-468e2e4858d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:48.177716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e1ad4-4d93-4291-ad33-468e2e4858d7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:48.185675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799e40a-3373-4fa3-b6ad-086269da3106 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-f15e1ad4-4d93-4291-ad33-468e2e4858d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:58:48.201352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:58:48.203842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:58:48.204777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:58:48.204777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:58:48.204777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:58:48.204777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 8/18] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:58:48.212272 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e1ad4-4d93-4291-ad33-468e2e4858d7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:48.238414 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed99c7-1496-4cac-9cc3-af3fbf6179b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:48.239539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed99c7-1496-4cac-9cc3-af3fbf6179b8 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:48.240031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed99c7-1496-4cac-9cc3-af3fbf6179b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:48.240521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed99c7-1496-4cac-9cc3-af3fbf6179b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:48.242694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed99c7-1496-4cac-9cc3-af3fbf6179b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:48.245445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed99c7-1496-4cac-9cc3-af3fbf6179b8 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:48.248493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 11/19] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:58:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:58:48.256158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed99c7-1496-4cac-9cc3-af3fbf6179b8 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:48.291706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:48.356414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e1ad4-4d93-4291-ad33-468e2e4858d7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:48.360871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:48.402137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42828d0b-985f-4769-8004-cbfb57644f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:48.402983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42828d0b-985f-4769-8004-cbfb57644f3d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:48.403421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42828d0b-985f-4769-8004-cbfb57644f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:48.403756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42828d0b-985f-4769-8004-cbfb57644f3d None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:48.404500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42828d0b-985f-4769-8004-cbfb57644f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:58:48.404723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed99c7-1496-4cac-9cc3-af3fbf6179b8 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:48.404881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42828d0b-985f-4769-8004-cbfb57644f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:48.407003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42828d0b-985f-4769-8004-cbfb57644f3d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:48.430713 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 9/20] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 12:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:58:48.457983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47fa99-2201-4abc-b28d-9537becaf28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:48.458953 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47fa99-2201-4abc-b28d-9537becaf28e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:48.459538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47fa99-2201-4abc-b28d-9537becaf28e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:48.459800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47fa99-2201-4abc-b28d-9537becaf28e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:48.461040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47fa99-2201-4abc-b28d-9537becaf28e None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.domain.id=default, user.id=8bfb2d4384094e9e8a8b910061e5c9d4) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:58:48.461512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47fa99-2201-4abc-b28d-9537becaf28e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:48.464042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47fa99-2201-4abc-b28d-9537becaf28e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:48.479951 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 12/21] 10.222.0.61 () {56 vars in 1322 bytes} [Wed Sep 7 12:58:48 2022] GET /identity/v3/role_assignments?user.id=8bfb2d4384094e9e8a8b910061e5c9d4&scope.domain.id=default&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 233 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:58:51.367517 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10d732-6f63-4136-a0a6-0628bf948739 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:51.368029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10d732-6f63-4136-a0a6-0628bf948739 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:51.368300 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10d732-6f63-4136-a0a6-0628bf948739 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:51.369745 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 10/22] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:51.385454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9552d-687d-41ff-82bb-4840299cf9b0 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:51.386435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9552d-687d-41ff-82bb-4840299cf9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:51.386943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9552d-687d-41ff-82bb-4840299cf9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:51.411535 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG passlib.handlers.bcrypt [None req-99f9552d-687d-41ff-82bb-4840299cf9b0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81796) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 12:58:51.412276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG passlib.handlers.bcrypt [None req-99f9552d-687d-41ff-82bb-4840299cf9b0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81796) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 12:58:51.458795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653afddd-1111-4e9a-9549-7d3472f7243e None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:51.459502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653afddd-1111-4e9a-9549-7d3472f7243e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:51.459997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653afddd-1111-4e9a-9549-7d3472f7243e None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:51.461950 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 11/23] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:58:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:58:51.467225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-99f9552d-687d-41ff-82bb-4840299cf9b0 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:51.468478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a60f69-c0b4-48de-ae13-a79ee86a21f0 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:51.468981 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a60f69-c0b4-48de-ae13-a79ee86a21f0 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:51.469478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a60f69-c0b4-48de-ae13-a79ee86a21f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:51.469580 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-99f9552d-687d-41ff-82bb-4840299cf9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:51.504746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-90a60f69-c0b4-48de-ae13-a79ee86a21f0 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:51.506361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-90a60f69-c0b4-48de-ae13-a79ee86a21f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:51.644617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 13/24] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:58:51.680562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 12/25] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:58:51.801291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3f3b1-e03e-4ed7-ab48-55892ecad070 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:51.802261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3f3b1-e03e-4ed7-ab48-55892ecad070 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:51.803012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3f3b1-e03e-4ed7-ab48-55892ecad070 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:51.806170 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 14/26] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:58:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:51.817763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79780974-3de1-440f-ab5b-531d4cd23956 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:51.818045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79780974-3de1-440f-ab5b-531d4cd23956 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:51.818369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79780974-3de1-440f-ab5b-531d4cd23956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:51.848401 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-79780974-3de1-440f-ab5b-531d4cd23956 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:51.849651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-79780974-3de1-440f-ab5b-531d4cd23956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:51.953158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 13/27] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:58:52.237025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3f3b1-e03e-4ed7-ab48-55892ecad070 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:52.241953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79780974-3de1-440f-ab5b-531d4cd23956 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:52.277223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:52.309168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:52.455096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79780974-3de1-440f-ab5b-531d4cd23956 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:52.478884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3f3b1-e03e-4ed7-ab48-55892ecad070 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:52.538715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af8cbe-0f95-4c30-bc18-4af44d2bed5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:52.539302 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af8cbe-0f95-4c30-bc18-4af44d2bed5c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:52.539576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af8cbe-0f95-4c30-bc18-4af44d2bed5c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:52.539837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af8cbe-0f95-4c30-bc18-4af44d2bed5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:52.552032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c069ef-39ea-4301-81d8-e5fd9f4f5978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:52.552777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c069ef-39ea-4301-81d8-e5fd9f4f5978 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:52.553070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c069ef-39ea-4301-81d8-e5fd9f4f5978 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:52.553308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c069ef-39ea-4301-81d8-e5fd9f4f5978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:52.570307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af8cbe-0f95-4c30-bc18-4af44d2bed5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-69af8cbe-0f95-4c30-bc18-4af44d2bed5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:58:52.582845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:58:52.584592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 14/28] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 12:58:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:58:52.586099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af8cbe-0f95-4c30-bc18-4af44d2bed5c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:52.599612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c069ef-39ea-4301-81d8-e5fd9f4f5978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:52.601362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c069ef-39ea-4301-81d8-e5fd9f4f5978 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-08c069ef-39ea-4301-81d8-e5fd9f4f5978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 12:58:52.646415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:58:52.648069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 12:58:52.649234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 12:58:52.649234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 12:58:52.649234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:58:52.649234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 15/29] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:58:52.653054 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c069ef-39ea-4301-81d8-e5fd9f4f5978 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:52.792201 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:52.844356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af8cbe-0f95-4c30-bc18-4af44d2bed5c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:52.852625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:52.915983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c069ef-39ea-4301-81d8-e5fd9f4f5978 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:52.944594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23544c-ab41-405c-beec-b505f65815e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:52.945733 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23544c-ab41-405c-beec-b505f65815e7 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:52.946144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23544c-ab41-405c-beec-b505f65815e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:52.946507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23544c-ab41-405c-beec-b505f65815e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:52.947307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23544c-ab41-405c-beec-b505f65815e7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:52.948871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23544c-ab41-405c-beec-b505f65815e7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:53.002144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e67356-cd23-44c5-ad60-81a0e4fb4640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:53.002785 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e67356-cd23-44c5-ad60-81a0e4fb4640 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:53.003064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e67356-cd23-44c5-ad60-81a0e4fb4640 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:53.003362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e67356-cd23-44c5-ad60-81a0e4fb4640 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:53.004319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e67356-cd23-44c5-ad60-81a0e4fb4640 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:58:53.004677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e67356-cd23-44c5-ad60-81a0e4fb4640 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:53.009763 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e67356-cd23-44c5-ad60-81a0e4fb4640 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:53.031259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 16/30] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:58:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:58:53.032341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e67356-cd23-44c5-ad60-81a0e4fb4640 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:53.192715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 15/31] 10.222.0.61 () {60 vars in 1131 bytes} [Wed Sep 7 12:58:52 2022] POST /identity/v3/roles => generated 225 bytes in 608 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:58:53.193797 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23544c-ab41-405c-beec-b505f65815e7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:53.268911 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e67356-cd23-44c5-ad60-81a0e4fb4640 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:53.366312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7ed39-097b-4975-8cd4-408300834fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:53.366876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7ed39-097b-4975-8cd4-408300834fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:53.367184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7ed39-097b-4975-8cd4-408300834fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:53.367564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7ed39-097b-4975-8cd4-408300834fe8 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:53.368339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7ed39-097b-4975-8cd4-408300834fe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:58:53.368652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7ed39-097b-4975-8cd4-408300834fe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:53.370141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7ed39-097b-4975-8cd4-408300834fe8 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:53.405180 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:53.409633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 17/32] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 12:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:58:53.418818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7ed39-097b-4975-8cd4-408300834fe8 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:53.441738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23544c-ab41-405c-beec-b505f65815e7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:53.530560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca38b5-abdf-4319-aacd-6e4deae6a9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:53.531557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca38b5-abdf-4319-aacd-6e4deae6a9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:53.531686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca38b5-abdf-4319-aacd-6e4deae6a9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:53.532025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca38b5-abdf-4319-aacd-6e4deae6a9d7 None admin] PATH_INFO: `/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:53.537459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca38b5-abdf-4319-aacd-6e4deae6a9d7 None admin] RBAC: Authorizing `identity:get_user(user_id=8bfb2d4384094e9e8a8b910061e5c9d4)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:53.539571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca38b5-abdf-4319-aacd-6e4deae6a9d7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:53.541344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 16/33] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:58:53 2022] GET /identity/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4 => generated 247 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:58:53.542726 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a01e4-a210-496e-948e-7eee591eaac0 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:53.543023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a01e4-a210-496e-948e-7eee591eaac0 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:53.546771 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a01e4-a210-496e-948e-7eee591eaac0 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:53.548794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 17/34] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:58:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:53.549660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a01e4-a210-496e-948e-7eee591eaac0 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:53.584715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:53.622842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7ed39-097b-4975-8cd4-408300834fe8 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:53.685094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19482a-4c03-4753-a0eb-66c5d290fa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:53.686127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19482a-4c03-4753-a0eb-66c5d290fa22 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:53.687321 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19482a-4c03-4753-a0eb-66c5d290fa22 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:53.687599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19482a-4c03-4753-a0eb-66c5d290fa22 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:53.689151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19482a-4c03-4753-a0eb-66c5d290fa22 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:53.691582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19482a-4c03-4753-a0eb-66c5d290fa22 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:53.749763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a01e4-a210-496e-948e-7eee591eaac0 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:53.810578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37757b84-87d3-4531-8037-3de83e88b3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:53.812090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37757b84-87d3-4531-8037-3de83e88b3af None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:53.812593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37757b84-87d3-4531-8037-3de83e88b3af None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:53.813092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37757b84-87d3-4531-8037-3de83e88b3af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:53.815505 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37757b84-87d3-4531-8037-3de83e88b3af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:53.818550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37757b84-87d3-4531-8037-3de83e88b3af None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:53.827513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 18/35] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:58:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:58:53.827513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c30c8-6193-4cb3-a8b5-e2d19ef2d984 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:53.827513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c30c8-6193-4cb3-a8b5-e2d19ef2d984 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:53.827513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c30c8-6193-4cb3-a8b5-e2d19ef2d984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:53.849010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 18/36] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 12:58:53 2022] POST /identity/v3/projects => generated 301 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:58:53.850888 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2f611-d441-4eb8-b5b5-a4519b092f8f None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:53.851293 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2f611-d441-4eb8-b5b5-a4519b092f8f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:53.851588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2f611-d441-4eb8-b5b5-a4519b092f8f None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:53.859225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 19/37] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:53.859225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2f611-d441-4eb8-b5b5-a4519b092f8f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:53.873843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ef9c30c8-6193-4cb3-a8b5-e2d19ef2d984 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:53.877089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ef9c30c8-6193-4cb3-a8b5-e2d19ef2d984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:53.973949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:53.990102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 19/38] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:58:53.992023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab1425-e60d-4144-8928-de57e1cfc4c2 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:53.992493 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab1425-e60d-4144-8928-de57e1cfc4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:53.992905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab1425-e60d-4144-8928-de57e1cfc4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:54.022708 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-a6ab1425-e60d-4144-8928-de57e1cfc4c2 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:54.024607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a6ab1425-e60d-4144-8928-de57e1cfc4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:54.028410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:54.090581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2f611-d441-4eb8-b5b5-a4519b092f8f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:54.120510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 20/39] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:58:54.121894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5a533-6632-4732-9a85-42ed16b73372 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:54.122290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5a533-6632-4732-9a85-42ed16b73372 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:54.122562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5a533-6632-4732-9a85-42ed16b73372 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:54.125093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 21/40] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:58:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:54.131218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fdc31-61b5-4080-bd48-d1e8de98d295 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:54.131647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fdc31-61b5-4080-bd48-d1e8de98d295 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:54.132038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fdc31-61b5-4080-bd48-d1e8de98d295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:54.172105 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fd0b3-ac6d-4d10-8750-fafbba4dea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:54.173656 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fd0b3-ac6d-4d10-8750-fafbba4dea00 None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:54.174436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fd0b3-ac6d-4d10-8750-fafbba4dea00 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:54.175176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fd0b3-ac6d-4d10-8750-fafbba4dea00 None admin] PATH_INFO: `/v3/domains/default/users/8bfb2d4384094e9e8a8b910061e5c9d4/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:54.180457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fd0b3-ac6d-4d10-8750-fafbba4dea00 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8bfb2d4384094e9e8a8b910061e5c9d4, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:54.184106 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fd0b3-ac6d-4d10-8750-fafbba4dea00 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:54.219321 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 20/41] 10.222.0.61 () {58 vars in 1361 bytes} [Wed Sep 7 12:58:53 2022] PUT /identity/v3/domains/default/users/8bfb2d4384094e9e8a8b910061e5c9d4/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:58:54.278713 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ef2fdc31-61b5-4080-bd48-d1e8de98d295 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:54.280195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ef2fdc31-61b5-4080-bd48-d1e8de98d295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:54.330497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fd0b3-ac6d-4d10-8750-fafbba4dea00 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:54.375856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 22/42] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:58:54.553425 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fd0b3-ac6d-4d10-8750-fafbba4dea00 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:54.572284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:54.601016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e032b-d993-4098-960f-03c358960051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:54.601656 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e032b-d993-4098-960f-03c358960051 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:54.601959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e032b-d993-4098-960f-03c358960051 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:54.602218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e032b-d993-4098-960f-03c358960051 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:54.603017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e032b-d993-4098-960f-03c358960051 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:54.604505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e032b-d993-4098-960f-03c358960051 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:54.623490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 21/43] 10.222.0.61 () {60 vars in 1131 bytes} [Wed Sep 7 12:58:54 2022] POST /identity/v3/roles => generated 223 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:58:54.676458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fdc31-61b5-4080-bd48-d1e8de98d295 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:54.727587 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e032b-d993-4098-960f-03c358960051 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:54.799668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:54.853124 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fdc31-61b5-4080-bd48-d1e8de98d295 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:54.900633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e032b-d993-4098-960f-03c358960051 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:54.911633 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35630f5a-73a5-4626-be59-6b1a41f691dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:54.912812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35630f5a-73a5-4626-be59-6b1a41f691dd None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:54.913276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35630f5a-73a5-4626-be59-6b1a41f691dd None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:54.913724 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35630f5a-73a5-4626-be59-6b1a41f691dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:54.915166 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35630f5a-73a5-4626-be59-6b1a41f691dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:54.917787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35630f5a-73a5-4626-be59-6b1a41f691dd None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:54.920353 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 23/44] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:58:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:58:54.926852 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35630f5a-73a5-4626-be59-6b1a41f691dd None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:54.955811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89df90a-901b-4da2-8212-72f01699877b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:54.956457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89df90a-901b-4da2-8212-72f01699877b None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:54.956744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89df90a-901b-4da2-8212-72f01699877b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:54.957026 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89df90a-901b-4da2-8212-72f01699877b None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:54.957742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89df90a-901b-4da2-8212-72f01699877b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:54.959218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89df90a-901b-4da2-8212-72f01699877b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:54.976752 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 22/45] 10.222.0.61 () {60 vars in 1131 bytes} [Wed Sep 7 12:58:54 2022] POST /identity/v3/roles => generated 219 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:58:55.067666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35630f5a-73a5-4626-be59-6b1a41f691dd None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:55.137688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10bcf5-16b4-41c0-81f0-c41312a92d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:55.138740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10bcf5-16b4-41c0-81f0-c41312a92d73 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:55.139279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10bcf5-16b4-41c0-81f0-c41312a92d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:55.139861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10bcf5-16b4-41c0-81f0-c41312a92d73 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:55.140842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10bcf5-16b4-41c0-81f0-c41312a92d73 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:55.143476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10bcf5-16b4-41c0-81f0-c41312a92d73 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:55.163186 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:55.200328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 24/46] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 12:58:54 2022] POST /identity/v3/projects => generated 312 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:58:56.762549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee80fef-d17e-4c78-ab92-5a8f1565c3d1 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:56.763140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee80fef-d17e-4c78-ab92-5a8f1565c3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:56.763140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee80fef-d17e-4c78-ab92-5a8f1565c3d1 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:56.764582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 23/47] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:58:56.770430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042328f-5f2b-449f-a8fa-28510887a07e None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:56.770981 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042328f-5f2b-449f-a8fa-28510887a07e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:56.771132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042328f-5f2b-449f-a8fa-28510887a07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:56.791886 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-d042328f-5f2b-449f-a8fa-28510887a07e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:56.793288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d042328f-5f2b-449f-a8fa-28510887a07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:56.870714 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 25/48] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:58:57.143459 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee80fef-d17e-4c78-ab92-5a8f1565c3d1 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:57.181918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:57.270526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee80fef-d17e-4c78-ab92-5a8f1565c3d1 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:57.318683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19b764-39d1-4073-871d-e08a937e6860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:57.319374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b764-39d1-4073-871d-e08a937e6860 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:57.319657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b764-39d1-4073-871d-e08a937e6860 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:57.319934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b764-39d1-4073-871d-e08a937e6860 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:57.332932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19b764-39d1-4073-871d-e08a937e6860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-fd19b764-39d1-4073-871d-e08a937e6860 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:58:57.338601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:58:57.340789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:58:57.342325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:58:57.342325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:58:57.342325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:58:57.342325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:58:57.342325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:58:57.342325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:58:57.342325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:58:57.342325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:58:57.342325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:58:57.342325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:58:57.342325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:58:57.342325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:58:57.342325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 24/49] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 12:58:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:58:57.352930 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042328f-5f2b-449f-a8fa-28510887a07e None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:57.501730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042328f-5f2b-449f-a8fa-28510887a07e None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:57.533801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:57.556735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9d813-350a-4149-a54b-1a6c1bb98ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:57.557782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9d813-350a-4149-a54b-1a6c1bb98ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:57.558245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9d813-350a-4149-a54b-1a6c1bb98ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:57.558714 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9d813-350a-4149-a54b-1a6c1bb98ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:57.560037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9d813-350a-4149-a54b-1a6c1bb98ad0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:58:57.560539 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9d813-350a-4149-a54b-1a6c1bb98ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:57.563759 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9d813-350a-4149-a54b-1a6c1bb98ad0 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:57.588188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 26/50] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:58:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:58:57.594212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19b764-39d1-4073-871d-e08a937e6860 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:57.685928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d7835-269c-4ad5-8514-e6c0a0414266 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:57.686521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d7835-269c-4ad5-8514-e6c0a0414266 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:57.686902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d7835-269c-4ad5-8514-e6c0a0414266 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:57.688803 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 27/51] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:58:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:58:57.695591 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c242a-a67c-4b8e-8ad4-25574da15998 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:57.695948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c242a-a67c-4b8e-8ad4-25574da15998 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:57.696367 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c242a-a67c-4b8e-8ad4-25574da15998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:57.719374 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-9f9c242a-a67c-4b8e-8ad4-25574da15998 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:58:57.720893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-9f9c242a-a67c-4b8e-8ad4-25574da15998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:58:57.733368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:57.745982 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19b764-39d1-4073-871d-e08a937e6860 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:57.797716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8162bd-59fe-4be2-8a2f-b2c9264b76f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:57.798724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8162bd-59fe-4be2-8a2f-b2c9264b76f4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:57.799174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8162bd-59fe-4be2-8a2f-b2c9264b76f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:57.799509 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8162bd-59fe-4be2-8a2f-b2c9264b76f4 None admin] PATH_INFO: `/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:57.801336 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8162bd-59fe-4be2-8a2f-b2c9264b76f4 None admin] RBAC: Authorizing `identity:get_user(user_id=8bfb2d4384094e9e8a8b910061e5c9d4)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:57.803916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8162bd-59fe-4be2-8a2f-b2c9264b76f4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:57.806619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 25/52] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:58:57 2022] GET /identity/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4 => generated 247 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:58:57.813525 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8162bd-59fe-4be2-8a2f-b2c9264b76f4 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:57.821744 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 28/53] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:58:57.925923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8162bd-59fe-4be2-8a2f-b2c9264b76f4 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:57.961740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07580ee1-20df-4fcd-85d6-4622f246ba43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:57.962389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07580ee1-20df-4fcd-85d6-4622f246ba43 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:57.962673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07580ee1-20df-4fcd-85d6-4622f246ba43 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:57.962938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07580ee1-20df-4fcd-85d6-4622f246ba43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:57.964109 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07580ee1-20df-4fcd-85d6-4622f246ba43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:57.965568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07580ee1-20df-4fcd-85d6-4622f246ba43 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:57.967167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 26/54] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:58:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:58:57.973693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c242a-a67c-4b8e-8ad4-25574da15998 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:58.096241 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07580ee1-20df-4fcd-85d6-4622f246ba43 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:58.100938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c242a-a67c-4b8e-8ad4-25574da15998 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:58.116536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:58.137955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:58:58.143356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3fc96-ff2e-4b42-8e2d-2935e1f2a706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:58.144417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3fc96-ff2e-4b42-8e2d-2935e1f2a706 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:58.144913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3fc96-ff2e-4b42-8e2d-2935e1f2a706 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:58.145423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3fc96-ff2e-4b42-8e2d-2935e1f2a706 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:58.146810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3fc96-ff2e-4b42-8e2d-2935e1f2a706 None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.domain.id=default, user.id=8bfb2d4384094e9e8a8b910061e5c9d4) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:58:58.147396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3fc96-ff2e-4b42-8e2d-2935e1f2a706 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:58.150130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3fc96-ff2e-4b42-8e2d-2935e1f2a706 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:58.164016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 29/55] 10.222.0.61 () {56 vars in 1322 bytes} [Wed Sep 7 12:58:57 2022] GET /identity/v3/role_assignments?user.id=8bfb2d4384094e9e8a8b910061e5c9d4&scope.domain.id=default&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 530 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:58:58.248614 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07580ee1-20df-4fcd-85d6-4622f246ba43 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:58.301908 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8f478-cb63-46aa-abee-e9fb6554c251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:58.302578 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8f478-cb63-46aa-abee-e9fb6554c251 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:58.302918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8f478-cb63-46aa-abee-e9fb6554c251 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:58.303166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8f478-cb63-46aa-abee-e9fb6554c251 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:58.304326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8f478-cb63-46aa-abee-e9fb6554c251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:58.305849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8f478-cb63-46aa-abee-e9fb6554c251 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:58.307552 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 27/56] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:58:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:58:58.314147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3fc96-ff2e-4b42-8e2d-2935e1f2a706 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:58:58.454368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3fc96-ff2e-4b42-8e2d-2935e1f2a706 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:58:58.514172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c07cb-1479-4143-92f2-2795c006fefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:58:58.515788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c07cb-1479-4143-92f2-2795c006fefb None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:58:58.516291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c07cb-1479-4143-92f2-2795c006fefb None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:58:58.516782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c07cb-1479-4143-92f2-2795c006fefb None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:58:58.518031 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c07cb-1479-4143-92f2-2795c006fefb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:58:58.520819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c07cb-1479-4143-92f2-2795c006fefb None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:58:58.566864 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 30/57] 10.222.0.61 () {60 vars in 1140 bytes} [Wed Sep 7 12:58:58 2022] POST /identity/v3/projects => generated 298 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:59:00.879735 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ded5f-b450-4b8b-8323-50cfcf4d8f64 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:00.880269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ded5f-b450-4b8b-8323-50cfcf4d8f64 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:00.880397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ded5f-b450-4b8b-8323-50cfcf4d8f64 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:00.881789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 28/58] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:59:00.888963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46faa374-654a-4a09-858e-b031509f84b2 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:00.889978 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46faa374-654a-4a09-858e-b031509f84b2 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:00.890438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46faa374-654a-4a09-858e-b031509f84b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:00.922102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-46faa374-654a-4a09-858e-b031509f84b2 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:00.924240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-46faa374-654a-4a09-858e-b031509f84b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:00.968370 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:01.024734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 31/59] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:01.303241 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ded5f-b450-4b8b-8323-50cfcf4d8f64 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:01.416469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:01.433582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ded5f-b450-4b8b-8323-50cfcf4d8f64 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:01.472950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac58fe-62a0-4461-85b9-0fb8cfd4c8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:01.473587 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac58fe-62a0-4461-85b9-0fb8cfd4c8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:01.473869 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac58fe-62a0-4461-85b9-0fb8cfd4c8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:01.474147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac58fe-62a0-4461-85b9-0fb8cfd4c8f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:01.475357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac58fe-62a0-4461-85b9-0fb8cfd4c8f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:01.476833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac58fe-62a0-4461-85b9-0fb8cfd4c8f1 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:01.478424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 29/60] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:59:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:01.484222 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46faa374-654a-4a09-858e-b031509f84b2 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:01.604432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46faa374-654a-4a09-858e-b031509f84b2 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:01.641149 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd79fd-0724-4d3e-a470-1c761b49eed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:01.641786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd79fd-0724-4d3e-a470-1c761b49eed5 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:01.642064 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd79fd-0724-4d3e-a470-1c761b49eed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:01.642337 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd79fd-0724-4d3e-a470-1c761b49eed5 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:01.643017 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd79fd-0724-4d3e-a470-1c761b49eed5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:01.644577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd79fd-0724-4d3e-a470-1c761b49eed5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:01.675310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 32/61] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 12:59:01 2022] POST /identity/v3/users => generated 275 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:59:05.059053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e03b90-0088-4132-a3c4-d76da51dae99 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:05.060192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e03b90-0088-4132-a3c4-d76da51dae99 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:05.060686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e03b90-0088-4132-a3c4-d76da51dae99 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:05.062650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 30/62] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:05.071910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f787644-9c8c-4227-81c5-f24b0eda0ffc None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:05.072869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f787644-9c8c-4227-81c5-f24b0eda0ffc None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:05.073364 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f787644-9c8c-4227-81c5-f24b0eda0ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:05.111902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-3f787644-9c8c-4227-81c5-f24b0eda0ffc None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:05.113574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3f787644-9c8c-4227-81c5-f24b0eda0ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:05.129522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:05.219659 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 33/63] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:05.561114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e03b90-0088-4132-a3c4-d76da51dae99 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:05.825647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7c2df-15d6-413b-b993-51e96abb48e7 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:05.826160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7c2df-15d6-413b-b993-51e96abb48e7 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:05.827172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7c2df-15d6-413b-b993-51e96abb48e7 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:05.828790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 34/64] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:59:05.840674 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58805ab-b18d-4376-a60a-3d1bb17f94ea None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:05.841211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58805ab-b18d-4376-a60a-3d1bb17f94ea None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:05.841637 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58805ab-b18d-4376-a60a-3d1bb17f94ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:05.862892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e03b90-0088-4132-a3c4-d76da51dae99 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:05.868069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-d58805ab-b18d-4376-a60a-3d1bb17f94ea None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:05.870815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d58805ab-b18d-4376-a60a-3d1bb17f94ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:05.905341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a906a-a922-4119-8ace-59840cef4a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:05.906000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a906a-a922-4119-8ace-59840cef4a4e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:05.906282 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a906a-a922-4119-8ace-59840cef4a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:05.906556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a906a-a922-4119-8ace-59840cef4a4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:05.918913 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a906a-a922-4119-8ace-59840cef4a4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-236a906a-a922-4119-8ace-59840cef4a4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:05.924831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:05.926974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:05.928520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:05.928520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:05.928520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:05.928520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:05.928520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:05.928520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:05.928520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:05.928520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:05.928520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:05.928520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:05.928520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:05.928520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:05.928520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 31/65] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:05.932939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3768994-bacb-45c6-bc59-b02ea8f86dea None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:05.933221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3768994-bacb-45c6-bc59-b02ea8f86dea None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:05.933473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3768994-bacb-45c6-bc59-b02ea8f86dea None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:05.935207 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 32/66] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:05.936147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3768994-bacb-45c6-bc59-b02ea8f86dea None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:05.977452 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:06.006692 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 35/67] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 07 12:59:06.009439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085de3a-7293-44d0-8187-c66b72d4f750 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:06.010275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085de3a-7293-44d0-8187-c66b72d4f750 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:06.010997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085de3a-7293-44d0-8187-c66b72d4f750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:06.041410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-b085de3a-7293-44d0-8187-c66b72d4f750 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:06.044097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b085de3a-7293-44d0-8187-c66b72d4f750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:06.058954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3768994-bacb-45c6-bc59-b02ea8f86dea None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:06.103814 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0e426-fce8-423c-90d3-def14b05fe39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:06.104737 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0e426-fce8-423c-90d3-def14b05fe39 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:06.104737 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0e426-fce8-423c-90d3-def14b05fe39 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:06.104984 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0e426-fce8-423c-90d3-def14b05fe39 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:06.105760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e0e426-fce8-423c-90d3-def14b05fe39 None admin] RBAC: Adding query filter params (name=member) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:06.106141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e0e426-fce8-423c-90d3-def14b05fe39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:06.107681 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e0e426-fce8-423c-90d3-def14b05fe39 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:06.119337 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 33/68] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:06.120963 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04de3b4-be5f-4d93-a6e4-ef888efba7d9 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:06.121374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04de3b4-be5f-4d93-a6e4-ef888efba7d9 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:06.121724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04de3b4-be5f-4d93-a6e4-ef888efba7d9 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:06.131029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 34/69] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:06.131029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04de3b4-be5f-4d93-a6e4-ef888efba7d9 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:06.178747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 36/70] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:06.180741 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2de7c-a0a8-42eb-a9e7-d5b02aa6b1ed None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:06.180741 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2de7c-a0a8-42eb-a9e7-d5b02aa6b1ed None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:06.180741 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2de7c-a0a8-42eb-a9e7-d5b02aa6b1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:06.189200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:06.202060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-a0b2de7c-a0a8-42eb-a9e7-d5b02aa6b1ed None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:06.203372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a0b2de7c-a0a8-42eb-a9e7-d5b02aa6b1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:06.283452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 37/71] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:06.310647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2de7c-a0a8-42eb-a9e7-d5b02aa6b1ed None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:06.322704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:06.358747 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04de3b4-be5f-4d93-a6e4-ef888efba7d9 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:06.469952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7ddd4-9c02-4d0a-a3bc-dbbd1ce6521e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:06.472502 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ddd4-9c02-4d0a-a3bc-dbbd1ce6521e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:06.473203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ddd4-9c02-4d0a-a3bc-dbbd1ce6521e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:06.473889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ddd4-9c02-4d0a-a3bc-dbbd1ce6521e None admin] PATH_INFO: `/v3/users/d7b269b260eb4a96aa4bc25ce7962e27` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:06.525752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2de7c-a0a8-42eb-a9e7-d5b02aa6b1ed None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:06.528216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7ddd4-9c02-4d0a-a3bc-dbbd1ce6521e None admin] RBAC: Authorizing `identity:get_user(user_id=d7b269b260eb4a96aa4bc25ce7962e27)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:06.531145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7ddd4-9c02-4d0a-a3bc-dbbd1ce6521e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:06.533194 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 35/72] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:06 2022] GET /identity/v3/users/d7b269b260eb4a96aa4bc25ce7962e27 => generated 275 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:06.539919 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc98305-71a3-4646-ab51-339d10164ac9 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:06.540349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc98305-71a3-4646-ab51-339d10164ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:06.540671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc98305-71a3-4646-ab51-339d10164ac9 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:06.550561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 36/73] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:06.551423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56ce81-ffed-4086-b0a4-664630a02f89 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:06.551715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56ce81-ffed-4086-b0a4-664630a02f89 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:06.551998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56ce81-ffed-4086-b0a4-664630a02f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:06.587000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1094ffe-be12-4df1-b643-b5e4c0862a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:06.587706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1094ffe-be12-4df1-b643-b5e4c0862a2e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:06.587989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1094ffe-be12-4df1-b643-b5e4c0862a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:06.588269 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1094ffe-be12-4df1-b643-b5e4c0862a2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:06.588955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-cf56ce81-ffed-4086-b0a4-664630a02f89 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:06.593266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-cf56ce81-ffed-4086-b0a4-664630a02f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:06.601483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1094ffe-be12-4df1-b643-b5e4c0862a2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-c1094ffe-be12-4df1-b643-b5e4c0862a2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:06.607000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:06.609552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:06.611414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:06.611414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:06.611414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:06.611414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:06.611414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:06.611414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:06.611414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:06.611414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:06.611414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:06.611414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:06.611414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:06.611414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:06.611414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 38/74] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:59:06.619366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1094ffe-be12-4df1-b643-b5e4c0862a2e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:06.619366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:06.741440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 37/75] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:06.743533 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56ce81-ffed-4086-b0a4-664630a02f89 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:06.745709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1094ffe-be12-4df1-b643-b5e4c0862a2e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:06.790156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d39be4-5827-4823-a851-7874ac5b109e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:06.790917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d39be4-5827-4823-a851-7874ac5b109e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:06.791240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d39be4-5827-4823-a851-7874ac5b109e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:06.791516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d39be4-5827-4823-a851-7874ac5b109e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:06.871211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:06.882983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d39be4-5827-4823-a851-7874ac5b109e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:06.886043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d39be4-5827-4823-a851-7874ac5b109e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:06.894004 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:06.931221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56ce81-ffed-4086-b0a4-664630a02f89 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:06.976280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d357f65-2a24-4243-8bf9-d7a4c5f7fb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:06.976924 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d357f65-2a24-4243-8bf9-d7a4c5f7fb9b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:06.977210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d357f65-2a24-4243-8bf9-d7a4c5f7fb9b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:06.977492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d357f65-2a24-4243-8bf9-d7a4c5f7fb9b None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:06.978201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d357f65-2a24-4243-8bf9-d7a4c5f7fb9b None admin] RBAC: Adding query filter params (name=member) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:06.978518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d357f65-2a24-4243-8bf9-d7a4c5f7fb9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:06.980159 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d357f65-2a24-4243-8bf9-d7a4c5f7fb9b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:06.991236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 38/76] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:06.996970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d357f65-2a24-4243-8bf9-d7a4c5f7fb9b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:07.022088 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 39/77] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:07.027470 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d39be4-5827-4823-a851-7874ac5b109e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:07.131212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d357f65-2a24-4243-8bf9-d7a4c5f7fb9b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:07.141917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:07.149456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d39be4-5827-4823-a851-7874ac5b109e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:07.188825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c90ab8-08a9-4451-8ef0-f6f9497e5099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:07.189894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c90ab8-08a9-4451-8ef0-f6f9497e5099 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:07.190194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c90ab8-08a9-4451-8ef0-f6f9497e5099 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:07.190461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c90ab8-08a9-4451-8ef0-f6f9497e5099 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:07.194630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d7969-780e-472d-9e52-4048355f073e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:07.195324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d7969-780e-472d-9e52-4048355f073e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:07.195616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d7969-780e-472d-9e52-4048355f073e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:07.196077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d7969-780e-472d-9e52-4048355f073e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:07.199935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:07.203177 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c90ab8-08a9-4451-8ef0-f6f9497e5099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-c3c90ab8-08a9-4451-8ef0-f6f9497e5099 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:07.208285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:07.209572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:07.210947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:07.210947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:07.210947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:07.210947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:07.210947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:07.210947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:07.210947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:07.210947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:07.210947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:07.210947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:07.210947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:59:07.210947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:07.210947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 40/78] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 12:59:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:07.212318 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d7969-780e-472d-9e52-4048355f073e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:07.214284 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c90ab8-08a9-4451-8ef0-f6f9497e5099 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-f18d7969-780e-472d-9e52-4048355f073e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:07.216394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:07.219717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 39/79] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 12:59:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:07.220489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d7969-780e-472d-9e52-4048355f073e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:07.338410 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d7969-780e-472d-9e52-4048355f073e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:07.338700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c90ab8-08a9-4451-8ef0-f6f9497e5099 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:07.382137 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687f5f1-27e1-4f83-891d-eb3d1fc93249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:07.382718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687f5f1-27e1-4f83-891d-eb3d1fc93249 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:07.383025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687f5f1-27e1-4f83-891d-eb3d1fc93249 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:07.383329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687f5f1-27e1-4f83-891d-eb3d1fc93249 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:07.384535 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687f5f1-27e1-4f83-891d-eb3d1fc93249 None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:07.386030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687f5f1-27e1-4f83-891d-eb3d1fc93249 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:07.387808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 40/80] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:07 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:07.388618 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687f5f1-27e1-4f83-891d-eb3d1fc93249 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:07.394544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e777e-dcec-4e00-b86f-dd2f5eac5350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:07.395433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e777e-dcec-4e00-b86f-dd2f5eac5350 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:07.395734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e777e-dcec-4e00-b86f-dd2f5eac5350 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:07.396009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e777e-dcec-4e00-b86f-dd2f5eac5350 None admin] PATH_INFO: `/v3/users/d7b269b260eb4a96aa4bc25ce7962e27` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:07.397241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e777e-dcec-4e00-b86f-dd2f5eac5350 None admin] RBAC: Authorizing `identity:get_user(user_id=d7b269b260eb4a96aa4bc25ce7962e27)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:07.398792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e777e-dcec-4e00-b86f-dd2f5eac5350 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:07.400615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 41/81] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:07 2022] GET /identity/v3/users/d7b269b260eb4a96aa4bc25ce7962e27 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:07.401452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e777e-dcec-4e00-b86f-dd2f5eac5350 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:07.428090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:07.487893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:07.519984 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e777e-dcec-4e00-b86f-dd2f5eac5350 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:07.540770 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687f5f1-27e1-4f83-891d-eb3d1fc93249 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:07.571320 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6babd-4d0e-44b5-b3fc-afd8410b45ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:07.572082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6babd-4d0e-44b5-b3fc-afd8410b45ce None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:07.572540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6babd-4d0e-44b5-b3fc-afd8410b45ce None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:07.572903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6babd-4d0e-44b5-b3fc-afd8410b45ce None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:07.573957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6babd-4d0e-44b5-b3fc-afd8410b45ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:07.574470 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6babd-4d0e-44b5-b3fc-afd8410b45ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:07.576293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6babd-4d0e-44b5-b3fc-afd8410b45ce None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:07.587175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa297ca-9a4c-4179-b697-5c3085373ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:07.588498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa297ca-9a4c-4179-b697-5c3085373ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:07.589007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 42/82] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:07.589262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa297ca-9a4c-4179-b697-5c3085373ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:07.590150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6babd-4d0e-44b5-b3fc-afd8410b45ce None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:07.591221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa297ca-9a4c-4179-b697-5c3085373ff3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:07.593787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa297ca-9a4c-4179-b697-5c3085373ff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:07.596242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa297ca-9a4c-4179-b697-5c3085373ff3 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:07.599150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 41/83] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:59:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:07.599703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa297ca-9a4c-4179-b697-5c3085373ff3 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:07.704006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:07.722622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6babd-4d0e-44b5-b3fc-afd8410b45ce None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:07.761267 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa297ca-9a4c-4179-b697-5c3085373ff3 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:07.767222 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe09a8-eaf6-41e5-983d-bc40b073a46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:07.768201 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe09a8-eaf6-41e5-983d-bc40b073a46f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:07.768656 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe09a8-eaf6-41e5-983d-bc40b073a46f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:07.769098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe09a8-eaf6-41e5-983d-bc40b073a46f None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:07.770213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe09a8-eaf6-41e5-983d-bc40b073a46f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:07.770707 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe09a8-eaf6-41e5-983d-bc40b073a46f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:07.773164 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe09a8-eaf6-41e5-983d-bc40b073a46f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:07.786346 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 43/84] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 12:59:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:07.787742 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe09a8-eaf6-41e5-983d-bc40b073a46f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:07.810538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:07.818860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6261132-e594-45d7-86c3-bdbf31fc144d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:07.819767 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6261132-e594-45d7-86c3-bdbf31fc144d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:07.820226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6261132-e594-45d7-86c3-bdbf31fc144d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:07.820675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6261132-e594-45d7-86c3-bdbf31fc144d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:07.821928 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6261132-e594-45d7-86c3-bdbf31fc144d None admin] RBAC: Adding query filter params (role.id=86f185569d134d80841142ae8e9d3f47, scope.project.id=81be097728134d76957fdb5755382cc2, user.id=d7b269b260eb4a96aa4bc25ce7962e27) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:07.822436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6261132-e594-45d7-86c3-bdbf31fc144d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:07.824653 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6261132-e594-45d7-86c3-bdbf31fc144d None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:07.833446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 42/85] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:07 2022] GET /identity/v3/role_assignments?user.id=d7b269b260eb4a96aa4bc25ce7962e27&scope.project.id=81be097728134d76957fdb5755382cc2&role.id=86f185569d134d80841142ae8e9d3f47 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:07.835227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6261132-e594-45d7-86c3-bdbf31fc144d None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:07.953516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe09a8-eaf6-41e5-983d-bc40b073a46f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:07.995320 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6261132-e594-45d7-86c3-bdbf31fc144d None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:08.019802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73e44b-2331-4648-aa3c-2262e24627e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:08.021214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73e44b-2331-4648-aa3c-2262e24627e8 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:08.021702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73e44b-2331-4648-aa3c-2262e24627e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:08.022178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73e44b-2331-4648-aa3c-2262e24627e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:08.037657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d0b0b-1c0f-4026-9065-a4ce8f82c2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:08.038335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d0b0b-1c0f-4026-9065-a4ce8f82c2a8 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:08.038618 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d0b0b-1c0f-4026-9065-a4ce8f82c2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:08.038894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d0b0b-1c0f-4026-9065-a4ce8f82c2a8 None admin] PATH_INFO: `/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:08.040221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d0b0b-1c0f-4026-9065-a4ce8f82c2a8 None admin] RBAC: Authorizing `identity:get_user(user_id=8bfb2d4384094e9e8a8b910061e5c9d4)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:08.041961 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d0b0b-1c0f-4026-9065-a4ce8f82c2a8 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:08.049349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 43/86] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:07 2022] GET /identity/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4 => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:08.049349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d0b0b-1c0f-4026-9065-a4ce8f82c2a8 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:08.058756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:08.152744 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73e44b-2331-4648-aa3c-2262e24627e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:08.154693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73e44b-2331-4648-aa3c-2262e24627e8 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:08.159243 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:08.218704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d0b0b-1c0f-4026-9065-a4ce8f82c2a8 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:08.263203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3bf0e-846c-4969-b03a-ec7b874226f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:08.264784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3bf0e-846c-4969-b03a-ec7b874226f4 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:08.265196 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3bf0e-846c-4969-b03a-ec7b874226f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:08.266024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3bf0e-846c-4969-b03a-ec7b874226f4 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:08.266700 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3bf0e-846c-4969-b03a-ec7b874226f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:08.268798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3bf0e-846c-4969-b03a-ec7b874226f4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:08.309518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 44/87] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 12:59:08 2022] POST /identity/v3/users => generated 289 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:59:08.311627 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3bf0e-846c-4969-b03a-ec7b874226f4 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:08.336654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 44/88] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:08.339285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73e44b-2331-4648-aa3c-2262e24627e8 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:08.433065 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:08.511858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3bf0e-846c-4969-b03a-ec7b874226f4 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:08.516841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73e44b-2331-4648-aa3c-2262e24627e8 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:08.551188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:08.582184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123364c8-3533-48ae-9a8e-752a5753ed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:08.583494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123364c8-3533-48ae-9a8e-752a5753ed59 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:08.584039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123364c8-3533-48ae-9a8e-752a5753ed59 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:08.584440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ccfc9-98ab-4a04-bc00-3de574439490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:08.584568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123364c8-3533-48ae-9a8e-752a5753ed59 None admin] PATH_INFO: `/v3/users/d7b269b260eb4a96aa4bc25ce7962e27` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:08.585423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ccfc9-98ab-4a04-bc00-3de574439490 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:08.586029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ccfc9-98ab-4a04-bc00-3de574439490 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:08.586465 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ccfc9-98ab-4a04-bc00-3de574439490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:08.586894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123364c8-3533-48ae-9a8e-752a5753ed59 None admin] RBAC: Authorizing `identity:get_user(user_id=d7b269b260eb4a96aa4bc25ce7962e27)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:08.589649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123364c8-3533-48ae-9a8e-752a5753ed59 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:08.593444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 45/89] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:08 2022] GET /identity/v3/users/d7b269b260eb4a96aa4bc25ce7962e27 => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:08.595673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123364c8-3533-48ae-9a8e-752a5753ed59 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:08.712538 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ccfc9-98ab-4a04-bc00-3de574439490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:08.715495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ccfc9-98ab-4a04-bc00-3de574439490 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:08.771463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123364c8-3533-48ae-9a8e-752a5753ed59 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:08.806038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:08.849840 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a27b0-dfc3-405e-967d-77c94a26971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:08.850899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a27b0-dfc3-405e-967d-77c94a26971f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:08.851329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a27b0-dfc3-405e-967d-77c94a26971f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:08.851772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a27b0-dfc3-405e-967d-77c94a26971f None admin] PATH_INFO: `/v3/projects/39584fb2ed764b26831eac9343a1d9c9` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:08.854051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a27b0-dfc3-405e-967d-77c94a26971f None admin] RBAC: Authorizing `identity:get_project(project_id=39584fb2ed764b26831eac9343a1d9c9)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:08.856561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a27b0-dfc3-405e-967d-77c94a26971f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:08.860392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 46/90] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:08 2022] GET /identity/v3/projects/39584fb2ed764b26831eac9343a1d9c9 => generated 312 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:08.863835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a27b0-dfc3-405e-967d-77c94a26971f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:08.926163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 45/91] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:08.928131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ccfc9-98ab-4a04-bc00-3de574439490 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:08.990285 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:09.054331 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ccfc9-98ab-4a04-bc00-3de574439490 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:09.082853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a27b0-dfc3-405e-967d-77c94a26971f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:09.097131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb4da1-ad3b-4524-8c9f-fd7fe61d9e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:09.097790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb4da1-ad3b-4524-8c9f-fd7fe61d9e1e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:09.098083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb4da1-ad3b-4524-8c9f-fd7fe61d9e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:09.098420 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb4da1-ad3b-4524-8c9f-fd7fe61d9e1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:09.099460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb4da1-ad3b-4524-8c9f-fd7fe61d9e1e None admin] RBAC: Adding query filter params (role.id=86f185569d134d80841142ae8e9d3f47, scope.project.id=39584fb2ed764b26831eac9343a1d9c9, user.id=d7b269b260eb4a96aa4bc25ce7962e27) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:09.099782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb4da1-ad3b-4524-8c9f-fd7fe61d9e1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:09.101341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb4da1-ad3b-4524-8c9f-fd7fe61d9e1e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:09.111243 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 46/92] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:08 2022] GET /identity/v3/role_assignments?user.id=d7b269b260eb4a96aa4bc25ce7962e27&scope.project.id=39584fb2ed764b26831eac9343a1d9c9&role.id=86f185569d134d80841142ae8e9d3f47 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:09.116371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb4da1-ad3b-4524-8c9f-fd7fe61d9e1e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:09.139020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:09.169969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6323a-3149-4dbf-969b-30d5e402b737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:09.171764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6323a-3149-4dbf-969b-30d5e402b737 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:09.172280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6323a-3149-4dbf-969b-30d5e402b737 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:09.172781 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6323a-3149-4dbf-969b-30d5e402b737 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:09.253537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb4da1-ad3b-4524-8c9f-fd7fe61d9e1e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:09.296224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5a4e3-be24-4e60-b44d-cf63d662221d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:09.296969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5a4e3-be24-4e60-b44d-cf63d662221d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:09.297261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5a4e3-be24-4e60-b44d-cf63d662221d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:09.297542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5a4e3-be24-4e60-b44d-cf63d662221d None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:09.298895 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5a4e3-be24-4e60-b44d-cf63d662221d None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:09.300497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5a4e3-be24-4e60-b44d-cf63d662221d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:09.302310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 47/93] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:09 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:09.314821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5a4e3-be24-4e60-b44d-cf63d662221d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:09.374093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6323a-3149-4dbf-969b-30d5e402b737 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:09.377669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6323a-3149-4dbf-969b-30d5e402b737 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:09.536409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:09.549198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:09.617866 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5a4e3-be24-4e60-b44d-cf63d662221d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:09.668476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 47/94] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 808 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:09.679726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6323a-3149-4dbf-969b-30d5e402b737 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:09.743988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389a779-d33e-4cad-8897-14e8da70239a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:09.744640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389a779-d33e-4cad-8897-14e8da70239a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:09.744929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389a779-d33e-4cad-8897-14e8da70239a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:09.745211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389a779-d33e-4cad-8897-14e8da70239a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:09.745987 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389a779-d33e-4cad-8897-14e8da70239a None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=81be097728134d76957fdb5755382cc2, user.id=8bfb2d4384094e9e8a8b910061e5c9d4) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:09.746447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389a779-d33e-4cad-8897-14e8da70239a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:09.747871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389a779-d33e-4cad-8897-14e8da70239a None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:09.765271 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 48/95] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:09 2022] GET /identity/v3/role_assignments?user.id=8bfb2d4384094e9e8a8b910061e5c9d4&scope.project.id=81be097728134d76957fdb5755382cc2&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 259 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:09.879565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6323a-3149-4dbf-969b-30d5e402b737 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:09.933664 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1f7e7-17f3-4d20-9af3-3903ba665c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:09.934338 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1f7e7-17f3-4d20-9af3-3903ba665c32 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:09.934704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1f7e7-17f3-4d20-9af3-3903ba665c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:09.934988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1f7e7-17f3-4d20-9af3-3903ba665c32 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:09.936924 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1f7e7-17f3-4d20-9af3-3903ba665c32 None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:09.938452 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1f7e7-17f3-4d20-9af3-3903ba665c32 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:09.940699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 48/96] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:09 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:09.949755 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389a779-d33e-4cad-8897-14e8da70239a None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:10.137526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389a779-d33e-4cad-8897-14e8da70239a None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:10.191938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e46db5-b22c-4971-b663-e4895044cdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:10.192601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e46db5-b22c-4971-b663-e4895044cdf3 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:10.192894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e46db5-b22c-4971-b663-e4895044cdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:10.193173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e46db5-b22c-4971-b663-e4895044cdf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:10.193951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e46db5-b22c-4971-b663-e4895044cdf3 None admin] RBAC: Adding query filter params (role.id=649721c72e5d40deabd3512c529ffdf4, scope.project.id=81be097728134d76957fdb5755382cc2, user.id=d7b269b260eb4a96aa4bc25ce7962e27) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:10.194269 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e46db5-b22c-4971-b663-e4895044cdf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:10.195944 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e46db5-b22c-4971-b663-e4895044cdf3 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:10.203435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 49/97] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:09 2022] GET /identity/v3/role_assignments?user.id=d7b269b260eb4a96aa4bc25ce7962e27&scope.project.id=81be097728134d76957fdb5755382cc2&role.id=649721c72e5d40deabd3512c529ffdf4 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:10.844431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43bce8-536f-4941-ad25-14cc32d87aa3 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:10.845186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43bce8-536f-4941-ad25-14cc32d87aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:10.845186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43bce8-536f-4941-ad25-14cc32d87aa3 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:10.846712 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 49/98] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:59:10.865155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaacf3-95d0-4f51-af45-d180bdb51869 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:10.865908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaacf3-95d0-4f51-af45-d180bdb51869 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:10.865908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaacf3-95d0-4f51-af45-d180bdb51869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:10.886864 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:10.900383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f8aaacf3-95d0-4f51-af45-d180bdb51869 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:10.901653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f8aaacf3-95d0-4f51-af45-d180bdb51869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:10.991895 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 50/99] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:11.591602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43bce8-536f-4941-ad25-14cc32d87aa3 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:11.627348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:11.755035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43bce8-536f-4941-ad25-14cc32d87aa3 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:11.802748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb624877-6f46-47c8-830e-100ea459e2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:11.803439 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb624877-6f46-47c8-830e-100ea459e2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:11.803722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb624877-6f46-47c8-830e-100ea459e2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:11.803998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb624877-6f46-47c8-830e-100ea459e2f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:11.824301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb624877-6f46-47c8-830e-100ea459e2f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-fb624877-6f46-47c8-830e-100ea459e2f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:11.830218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:11.831823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:11.833383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:11.833383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:11.833383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:11.833383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:11.833383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:11.833383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:11.833383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:11.833383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:11.833383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:11.833383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:11.833383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:11.833383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:11.833383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 50/100] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:59:11.845396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaacf3-95d0-4f51-af45-d180bdb51869 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:11.988763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaacf3-95d0-4f51-af45-d180bdb51869 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:12.043433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed97f8-4bc1-41b3-9bc2-7d0f13dd8328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:12.044094 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed97f8-4bc1-41b3-9bc2-7d0f13dd8328 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:12.044441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed97f8-4bc1-41b3-9bc2-7d0f13dd8328 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:12.044726 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed97f8-4bc1-41b3-9bc2-7d0f13dd8328 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:12.045731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed97f8-4bc1-41b3-9bc2-7d0f13dd8328 None admin] RBAC: Adding query filter params (name=member) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:12.046122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed97f8-4bc1-41b3-9bc2-7d0f13dd8328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:12.047677 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed97f8-4bc1-41b3-9bc2-7d0f13dd8328 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:12.057171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:12.065647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 51/101] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:12.095137 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb624877-6f46-47c8-830e-100ea459e2f3 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:12.231471 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:12.252191 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb624877-6f46-47c8-830e-100ea459e2f3 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:12.299689 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c79a5-2789-4315-adc6-fe88bd449d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:12.300355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c79a5-2789-4315-adc6-fe88bd449d37 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:12.300643 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c79a5-2789-4315-adc6-fe88bd449d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:12.300922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c79a5-2789-4315-adc6-fe88bd449d37 None admin] PATH_INFO: `/v3/users/d7b269b260eb4a96aa4bc25ce7962e27` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:12.306402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c79a5-2789-4315-adc6-fe88bd449d37 None admin] RBAC: Authorizing `identity:get_user(user_id=d7b269b260eb4a96aa4bc25ce7962e27)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:12.308138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c79a5-2789-4315-adc6-fe88bd449d37 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:12.309899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 51/102] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:12 2022] GET /identity/v3/users/d7b269b260eb4a96aa4bc25ce7962e27 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:12.319216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed97f8-4bc1-41b3-9bc2-7d0f13dd8328 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:12.474445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed97f8-4bc1-41b3-9bc2-7d0f13dd8328 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:12.525246 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f705c-efc7-4b40-adc4-e8bb94307006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:12.526015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f705c-efc7-4b40-adc4-e8bb94307006 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:12.526304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f705c-efc7-4b40-adc4-e8bb94307006 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:12.526612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f705c-efc7-4b40-adc4-e8bb94307006 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:12.532181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f705c-efc7-4b40-adc4-e8bb94307006 None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:12.533699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f705c-efc7-4b40-adc4-e8bb94307006 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:12.535843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 52/103] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:12 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:12.544804 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c79a5-2789-4315-adc6-fe88bd449d37 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:12.693312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c79a5-2789-4315-adc6-fe88bd449d37 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:12.743679 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bd4d5-f60b-406d-8f58-3a81e324c89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:12.744374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bd4d5-f60b-406d-8f58-3a81e324c89d None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:12.744658 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bd4d5-f60b-406d-8f58-3a81e324c89d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:12.744941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bd4d5-f60b-406d-8f58-3a81e324c89d None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2/users/d7b269b260eb4a96aa4bc25ce7962e27/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:12.752185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bd4d5-f60b-406d-8f58-3a81e324c89d None admin] RBAC: Authorizing `identity:create_grant(project_id=81be097728134d76957fdb5755382cc2, user_id=d7b269b260eb4a96aa4bc25ce7962e27, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:12.754127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bd4d5-f60b-406d-8f58-3a81e324c89d None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:12.771985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 52/104] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 12:59:12 2022] PUT /identity/v3/projects/81be097728134d76957fdb5755382cc2/users/d7b269b260eb4a96aa4bc25ce7962e27/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 12:59:13.157501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14626929-f3ad-44a8-8543-abe5dd67289b None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:13.158302 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14626929-f3ad-44a8-8543-abe5dd67289b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:13.158302 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14626929-f3ad-44a8-8543-abe5dd67289b None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:13.159491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 53/105] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:13.165498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce87f5a-e8e1-4c51-9e76-ae009373cd1a None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:13.166245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce87f5a-e8e1-4c51-9e76-ae009373cd1a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:13.166245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce87f5a-e8e1-4c51-9e76-ae009373cd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:13.196045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-4ce87f5a-e8e1-4c51-9e76-ae009373cd1a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:13.197323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4ce87f5a-e8e1-4c51-9e76-ae009373cd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:13.210617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:13.278051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701391d-ee7a-4c6b-b436-b73caa6dadbe None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:13.278677 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701391d-ee7a-4c6b-b436-b73caa6dadbe None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:13.279219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701391d-ee7a-4c6b-b436-b73caa6dadbe None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:13.280827 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 54/106] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:59:13.291333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056b7d5-dc3f-491f-ab23-187d350b5a38 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:13.291936 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056b7d5-dc3f-491f-ab23-187d350b5a38 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:13.292409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056b7d5-dc3f-491f-ab23-187d350b5a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:13.305233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 53/107] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:13.332226 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-1056b7d5-dc3f-491f-ab23-187d350b5a38 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:13.333760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1056b7d5-dc3f-491f-ab23-187d350b5a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:13.426485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:13.435342 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 55/108] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:59:13.824827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce87f5a-e8e1-4c51-9e76-ae009373cd1a None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:13.877116 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056b7d5-dc3f-491f-ab23-187d350b5a38 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:14.049098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce87f5a-e8e1-4c51-9e76-ae009373cd1a None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:14.136419 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef66b7c-6ee8-4421-93e1-fdf516d7deaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:14.137891 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef66b7c-6ee8-4421-93e1-fdf516d7deaa None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.138601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef66b7c-6ee8-4421-93e1-fdf516d7deaa None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.139894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef66b7c-6ee8-4421-93e1-fdf516d7deaa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.142332 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef66b7c-6ee8-4421-93e1-fdf516d7deaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:14.145583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef66b7c-6ee8-4421-93e1-fdf516d7deaa None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:14.148759 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 54/109] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:59:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:14.149106 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056b7d5-dc3f-491f-ab23-187d350b5a38 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:14.155939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b023d-969a-4472-bbea-c9e0dccc0962 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.156252 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b023d-969a-4472-bbea-c9e0dccc0962 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.156816 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b023d-969a-4472-bbea-c9e0dccc0962 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.159088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 55/110] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:14.160152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b023d-969a-4472-bbea-c9e0dccc0962 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:14.200866 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:14.214647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bf750-4f45-415e-b64d-475a635cbc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:14.215329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bf750-4f45-415e-b64d-475a635cbc16 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.215615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bf750-4f45-415e-b64d-475a635cbc16 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.215887 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bf750-4f45-415e-b64d-475a635cbc16 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.238748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bf750-4f45-415e-b64d-475a635cbc16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-4f5bf750-4f45-415e-b64d-475a635cbc16 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:14.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:14.250481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:14.251266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:14.251266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:14.251266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:14.251266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:14.251266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:14.251266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:14.251266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:14.251266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:14.251266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:14.251266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:14.251266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:14.251266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:14.253128 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 56/111] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:14.254739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002edfe-ea9c-4127-82fd-20d4a4ee72d9 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.255028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002edfe-ea9c-4127-82fd-20d4a4ee72d9 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.262526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002edfe-ea9c-4127-82fd-20d4a4ee72d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.324612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-a002edfe-ea9c-4127-82fd-20d4a4ee72d9 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:14.327057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a002edfe-ea9c-4127-82fd-20d4a4ee72d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:14.393328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b023d-969a-4472-bbea-c9e0dccc0962 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:14.462304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6b7a6-08a4-4e67-9c06-ea887c5f1f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:14.464414 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6b7a6-08a4-4e67-9c06-ea887c5f1f09 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.464908 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6b7a6-08a4-4e67-9c06-ea887c5f1f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.465395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6b7a6-08a4-4e67-9c06-ea887c5f1f09 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.466603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6b7a6-08a4-4e67-9c06-ea887c5f1f09 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:14.470041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6b7a6-08a4-4e67-9c06-ea887c5f1f09 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:14.542737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:14.554628 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 56/112] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 12:59:14 2022] POST /identity/v3/projects => generated 302 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:59:14.557699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6b7a6-08a4-4e67-9c06-ea887c5f1f09 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:14.575287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 57/113] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:14.577353 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443899fd-6784-47cf-96ba-3703d2e05219 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.577925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443899fd-6784-47cf-96ba-3703d2e05219 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.578457 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443899fd-6784-47cf-96ba-3703d2e05219 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.580442 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 58/114] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:14.586804 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946b05c-3b1d-4663-9c37-95c35c570334 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.587515 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946b05c-3b1d-4663-9c37-95c35c570334 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.588056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946b05c-3b1d-4663-9c37-95c35c570334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.610776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-2946b05c-3b1d-4663-9c37-95c35c570334 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:14.612397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2946b05c-3b1d-4663-9c37-95c35c570334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:14.682839 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:14.705456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 59/115] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:59:14.756411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6b7a6-08a4-4e67-9c06-ea887c5f1f09 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:14.807208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b83927-d84f-4a90-8185-83d720e9ccbf None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.807865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b83927-d84f-4a90-8185-83d720e9ccbf None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.808373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b83927-d84f-4a90-8185-83d720e9ccbf None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.810168 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 60/116] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:59:14.818668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e6c69-29e3-4ce4-985d-873cde542b52 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.819358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e6c69-29e3-4ce4-985d-873cde542b52 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.819840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e6c69-29e3-4ce4-985d-873cde542b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.820276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7805d2-cec2-410d-91ba-428c70adc61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:14.821233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7805d2-cec2-410d-91ba-428c70adc61e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:14.821660 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7805d2-cec2-410d-91ba-428c70adc61e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:14.822077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7805d2-cec2-410d-91ba-428c70adc61e None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:14.823231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7805d2-cec2-410d-91ba-428c70adc61e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:14.827152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7805d2-cec2-410d-91ba-428c70adc61e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:14.827152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7805d2-cec2-410d-91ba-428c70adc61e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:14.842481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 57/117] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:14.853420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7805d2-cec2-410d-91ba-428c70adc61e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:14.889338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-4d2e6c69-29e3-4ce4-985d-873cde542b52 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:14.891856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-4d2e6c69-29e3-4ce4-985d-873cde542b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:15.096403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 61/118] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:59:15.097311 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e6c69-29e3-4ce4-985d-873cde542b52 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:15.119519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7805d2-cec2-410d-91ba-428c70adc61e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:15.156804 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:15.164353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:15.186799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff3981-4838-4683-acfb-d632fa61298b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:15.188426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff3981-4838-4683-acfb-d632fa61298b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:15.189046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff3981-4838-4683-acfb-d632fa61298b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:15.189639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff3981-4838-4683-acfb-d632fa61298b None admin] PATH_INFO: `/v3/users/328674bcbff943fb878160494186b1b4` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:15.215528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff3981-4838-4683-acfb-d632fa61298b None admin] RBAC: Authorizing `identity:get_user(user_id=328674bcbff943fb878160494186b1b4)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:15.219861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff3981-4838-4683-acfb-d632fa61298b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:15.224114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 58/119] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:14 2022] GET /identity/v3/users/328674bcbff943fb878160494186b1b4 => generated 289 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:15.227704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e6c69-29e3-4ce4-985d-873cde542b52 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:15.231074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff3981-4838-4683-acfb-d632fa61298b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:15.289280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b7049-acc9-47ed-9c37-e9921e5b0e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:15.290454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b7049-acc9-47ed-9c37-e9921e5b0e19 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:15.291295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b7049-acc9-47ed-9c37-e9921e5b0e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:15.292847 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b7049-acc9-47ed-9c37-e9921e5b0e19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:15.315742 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b7049-acc9-47ed-9c37-e9921e5b0e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-5a9b7049-acc9-47ed-9c37-e9921e5b0e19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:15.322521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:15.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:15.325518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:15.325518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:15.325518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:15.325518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:15.325518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:15.325518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:15.325518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:15.325518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:15.325518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:15.325518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:15.325518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:15.325518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:15.325518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 62/120] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 12:59:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:15.334352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b7049-acc9-47ed-9c37-e9921e5b0e19 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:15.438083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff3981-4838-4683-acfb-d632fa61298b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:15.483176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b7049-acc9-47ed-9c37-e9921e5b0e19 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:15.501807 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:15.520776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8f590-8394-4c00-a0a5-4c3afa825b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:15.522337 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8f590-8394-4c00-a0a5-4c3afa825b54 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:15.523152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8f590-8394-4c00-a0a5-4c3afa825b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:15.523890 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8f590-8394-4c00-a0a5-4c3afa825b54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:15.606722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a1a54-14b2-435f-a9d2-a3ec3b96ba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:15.607380 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a1a54-14b2-435f-a9d2-a3ec3b96ba44 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:15.607616 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a1a54-14b2-435f-a9d2-a3ec3b96ba44 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:15.607890 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a1a54-14b2-435f-a9d2-a3ec3b96ba44 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:15.608585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a1a54-14b2-435f-a9d2-a3ec3b96ba44 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:15.608889 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a1a54-14b2-435f-a9d2-a3ec3b96ba44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:15.610418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a1a54-14b2-435f-a9d2-a3ec3b96ba44 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:15.632347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 63/121] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 12:59:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:15.633282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a1a54-14b2-435f-a9d2-a3ec3b96ba44 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:15.656446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:15.680113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8f590-8394-4c00-a0a5-4c3afa825b54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:15.683430 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8f590-8394-4c00-a0a5-4c3afa825b54 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:15.910145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a1a54-14b2-435f-a9d2-a3ec3b96ba44 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:15.948213 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 59/122] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:15.950331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8f590-8394-4c00-a0a5-4c3afa825b54 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:15.961832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94517263-56b1-4829-a74f-37184040bf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:15.963472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94517263-56b1-4829-a74f-37184040bf7c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:15.964248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94517263-56b1-4829-a74f-37184040bf7c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:15.964997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94517263-56b1-4829-a74f-37184040bf7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:15.986726 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94517263-56b1-4829-a74f-37184040bf7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-94517263-56b1-4829-a74f-37184040bf7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:15.998992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:16.000929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:16.002474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:16.002474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:16.002474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:16.002474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:16.002474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:16.002474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:16.002474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:16.002474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:16.002474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:16.002474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:16.002474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:16.002474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:16.006788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 64/123] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:16.013039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94517263-56b1-4829-a74f-37184040bf7c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:16.080977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:16.133647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:16.183876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8f590-8394-4c00-a0a5-4c3afa825b54 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:16.241080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94517263-56b1-4829-a74f-37184040bf7c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:16.257155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41ab82-57da-4a5a-b3ac-7ff17d4ff74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:16.257982 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41ab82-57da-4a5a-b3ac-7ff17d4ff74c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:16.258250 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41ab82-57da-4a5a-b3ac-7ff17d4ff74c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:16.258602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41ab82-57da-4a5a-b3ac-7ff17d4ff74c None admin] PATH_INFO: `/v3/users/d7b269b260eb4a96aa4bc25ce7962e27` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:16.260199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41ab82-57da-4a5a-b3ac-7ff17d4ff74c None admin] RBAC: Authorizing `identity:get_user(user_id=d7b269b260eb4a96aa4bc25ce7962e27)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:16.261952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41ab82-57da-4a5a-b3ac-7ff17d4ff74c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:16.264725 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 60/124] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:15 2022] GET /identity/v3/users/d7b269b260eb4a96aa4bc25ce7962e27 => generated 275 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:16.266001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41ab82-57da-4a5a-b3ac-7ff17d4ff74c None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:16.313855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed76ea-9851-4408-8343-73ae3304a7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:16.315445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed76ea-9851-4408-8343-73ae3304a7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:16.316206 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed76ea-9851-4408-8343-73ae3304a7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:16.317091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed76ea-9851-4408-8343-73ae3304a7f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:16.339351 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed76ea-9851-4408-8343-73ae3304a7f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-48ed76ea-9851-4408-8343-73ae3304a7f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:16.352023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:16.353790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:16.355373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:16.355373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:16.355373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:16.355373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:16.355373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:16.355373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:16.355373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:16.355373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:16.355373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:16.355373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:16.355373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:59:16.355373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:16.355373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 65/125] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 12:59:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:16.356674 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed76ea-9851-4408-8343-73ae3304a7f2 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:16.409928 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41ab82-57da-4a5a-b3ac-7ff17d4ff74c None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:16.455461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce48d15-e88b-422f-aea7-5a62e1d5d0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:16.456019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce48d15-e88b-422f-aea7-5a62e1d5d0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:16.456307 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce48d15-e88b-422f-aea7-5a62e1d5d0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:16.456591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce48d15-e88b-422f-aea7-5a62e1d5d0f0 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:16.457844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce48d15-e88b-422f-aea7-5a62e1d5d0f0 None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:16.459615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce48d15-e88b-422f-aea7-5a62e1d5d0f0 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:16.470197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 61/126] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:16 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:16.470197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce48d15-e88b-422f-aea7-5a62e1d5d0f0 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:16.496994 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:16.525280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:16.549361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed76ea-9851-4408-8343-73ae3304a7f2 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:16.614200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f41bac-5098-46b8-bf12-545ad2e55f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:16.615301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41bac-5098-46b8-bf12-545ad2e55f4a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:16.615794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41bac-5098-46b8-bf12-545ad2e55f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:16.616280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41bac-5098-46b8-bf12-545ad2e55f4a None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:16.617502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f41bac-5098-46b8-bf12-545ad2e55f4a None admin] RBAC: Adding query filter params (name=member) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:16.618025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f41bac-5098-46b8-bf12-545ad2e55f4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:16.620691 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f41bac-5098-46b8-bf12-545ad2e55f4a None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:16.636392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 66/127] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:16.638455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b573fa4-fd97-45a9-b909-61437328dba3 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:16.638999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b573fa4-fd97-45a9-b909-61437328dba3 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:16.639501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b573fa4-fd97-45a9-b909-61437328dba3 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:16.648221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 67/128] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:16.649413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b573fa4-fd97-45a9-b909-61437328dba3 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:16.652773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce48d15-e88b-422f-aea7-5a62e1d5d0f0 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:16.701470 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a06b8-3e34-47c9-9acb-880d4598fdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:16.702145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a06b8-3e34-47c9-9acb-880d4598fdbd None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:16.702433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a06b8-3e34-47c9-9acb-880d4598fdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:16.702717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a06b8-3e34-47c9-9acb-880d4598fdbd None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:16.704022 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a06b8-3e34-47c9-9acb-880d4598fdbd None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:16.705675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a06b8-3e34-47c9-9acb-880d4598fdbd None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:16.714853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 62/129] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:16 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:16.714853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a06b8-3e34-47c9-9acb-880d4598fdbd None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:16.836786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b573fa4-fd97-45a9-b909-61437328dba3 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:16.851723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:16.876229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a06b8-3e34-47c9-9acb-880d4598fdbd None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:16.895202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80491a09-c2b3-487d-9122-4b0e60595c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:16.896253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80491a09-c2b3-487d-9122-4b0e60595c51 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:16.896672 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80491a09-c2b3-487d-9122-4b0e60595c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:16.896994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80491a09-c2b3-487d-9122-4b0e60595c51 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:16.898206 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80491a09-c2b3-487d-9122-4b0e60595c51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:16.898487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80491a09-c2b3-487d-9122-4b0e60595c51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:16.900965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80491a09-c2b3-487d-9122-4b0e60595c51 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:16.921734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 68/130] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:16.921734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80491a09-c2b3-487d-9122-4b0e60595c51 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:16.939229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2292d13-70db-4b30-938f-a648cd25236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:16.939855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2292d13-70db-4b30-938f-a648cd25236c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:16.940151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2292d13-70db-4b30-938f-a648cd25236c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:16.940425 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2292d13-70db-4b30-938f-a648cd25236c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:16.941166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2292d13-70db-4b30-938f-a648cd25236c None admin] RBAC: Adding query filter params (role.id=4b22e77ca0b449f3b83e08e32d811efb, scope.project.id=81be097728134d76957fdb5755382cc2, user.id=328674bcbff943fb878160494186b1b4) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:16.941475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2292d13-70db-4b30-938f-a648cd25236c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:16.942806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:16.943083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2292d13-70db-4b30-938f-a648cd25236c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:16.949295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 63/131] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:16 2022] GET /identity/v3/role_assignments?user.id=328674bcbff943fb878160494186b1b4&scope.project.id=81be097728134d76957fdb5755382cc2&role.id=4b22e77ca0b449f3b83e08e32d811efb => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:16.950571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b5dea-1f55-444f-870c-ac94b6a948f1 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:16.950851 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b5dea-1f55-444f-870c-ac94b6a948f1 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:16.951269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b5dea-1f55-444f-870c-ac94b6a948f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:16.980666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b01b5dea-1f55-444f-870c-ac94b6a948f1 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:16.984355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b01b5dea-1f55-444f-870c-ac94b6a948f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:17.126044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80491a09-c2b3-487d-9122-4b0e60595c51 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:17.132334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 64/132] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:17.133718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b5dea-1f55-444f-870c-ac94b6a948f1 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:17.180718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7a17a-b1a2-435d-aadd-bd42a10a0db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:17.182037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7a17a-b1a2-435d-aadd-bd42a10a0db5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:17.182687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7a17a-b1a2-435d-aadd-bd42a10a0db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:17.183359 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7a17a-b1a2-435d-aadd-bd42a10a0db5 None admin] PATH_INFO: `/v3/users/d7b269b260eb4a96aa4bc25ce7962e27` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:17.185860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7a17a-b1a2-435d-aadd-bd42a10a0db5 None admin] RBAC: Authorizing `identity:get_user(user_id=d7b269b260eb4a96aa4bc25ce7962e27)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:17.189301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7a17a-b1a2-435d-aadd-bd42a10a0db5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:17.193870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 69/133] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:16 2022] GET /identity/v3/users/d7b269b260eb4a96aa4bc25ce7962e27 => generated 275 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:17.200402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7a17a-b1a2-435d-aadd-bd42a10a0db5 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:17.285659 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:17.332502 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b5dea-1f55-444f-870c-ac94b6a948f1 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:17.379264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:17.414681 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7a17a-b1a2-435d-aadd-bd42a10a0db5 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:17.479089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28985a30-59e6-4013-927f-3bf78d01464e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:17.479951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28985a30-59e6-4013-927f-3bf78d01464e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:17.480344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28985a30-59e6-4013-927f-3bf78d01464e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:17.480721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28985a30-59e6-4013-927f-3bf78d01464e None admin] PATH_INFO: `/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:17.482167 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28985a30-59e6-4013-927f-3bf78d01464e None admin] RBAC: Authorizing `identity:get_user(user_id=8bfb2d4384094e9e8a8b910061e5c9d4)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:17.484534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28985a30-59e6-4013-927f-3bf78d01464e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:17.484741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c36c91-2394-4796-acfb-91518fd81ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:17.485404 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c36c91-2394-4796-acfb-91518fd81ab6 None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:17.485689 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c36c91-2394-4796-acfb-91518fd81ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:17.485969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c36c91-2394-4796-acfb-91518fd81ab6 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2/users/d7b269b260eb4a96aa4bc25ce7962e27/roles/649721c72e5d40deabd3512c529ffdf4` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:17.487128 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 70/134] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:17 2022] GET /identity/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4 => generated 247 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:17.488083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28985a30-59e6-4013-927f-3bf78d01464e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:17.497041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c36c91-2394-4796-acfb-91518fd81ab6 None admin] RBAC: Authorizing `identity:create_grant(project_id=81be097728134d76957fdb5755382cc2, user_id=d7b269b260eb4a96aa4bc25ce7962e27, role_id=649721c72e5d40deabd3512c529ffdf4)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:17.498856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c36c91-2394-4796-acfb-91518fd81ab6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:17.537564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 65/135] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 12:59:17 2022] PUT /identity/v3/projects/81be097728134d76957fdb5755382cc2/users/d7b269b260eb4a96aa4bc25ce7962e27/roles/649721c72e5d40deabd3512c529ffdf4 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:59:17.538346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c36c91-2394-4796-acfb-91518fd81ab6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:17.619429 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28985a30-59e6-4013-927f-3bf78d01464e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:17.663129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad3f6d-12bd-4f04-aac9-cb7533bbdaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:17.663912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad3f6d-12bd-4f04-aac9-cb7533bbdaf9 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:17.664195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad3f6d-12bd-4f04-aac9-cb7533bbdaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:17.664531 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad3f6d-12bd-4f04-aac9-cb7533bbdaf9 None admin] PATH_INFO: `/v3/projects/39584fb2ed764b26831eac9343a1d9c9` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:17.665816 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad3f6d-12bd-4f04-aac9-cb7533bbdaf9 None admin] RBAC: Authorizing `identity:get_project(project_id=39584fb2ed764b26831eac9343a1d9c9)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:17.667721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad3f6d-12bd-4f04-aac9-cb7533bbdaf9 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:17.669898 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 71/136] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:17 2022] GET /identity/v3/projects/39584fb2ed764b26831eac9343a1d9c9 => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:17.676403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad3f6d-12bd-4f04-aac9-cb7533bbdaf9 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:17.688929 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c36c91-2394-4796-acfb-91518fd81ab6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:17.708492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:17.734644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53f6c4-2b39-4849-9902-bfdd84fa2356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:17.736937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53f6c4-2b39-4849-9902-bfdd84fa2356 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:17.738512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53f6c4-2b39-4849-9902-bfdd84fa2356 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:17.742097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53f6c4-2b39-4849-9902-bfdd84fa2356 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:17.745070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53f6c4-2b39-4849-9902-bfdd84fa2356 None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:17.749238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53f6c4-2b39-4849-9902-bfdd84fa2356 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:17.754779 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 66/137] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:17 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:17.765874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53f6c4-2b39-4849-9902-bfdd84fa2356 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:17.794719 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:17.829364 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad3f6d-12bd-4f04-aac9-cb7533bbdaf9 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:17.925441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2a4f7-db5a-420c-a9d8-0258cbe80df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:17.926147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2a4f7-db5a-420c-a9d8-0258cbe80df9 None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:17.926433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2a4f7-db5a-420c-a9d8-0258cbe80df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:17.926753 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2a4f7-db5a-420c-a9d8-0258cbe80df9 None admin] PATH_INFO: `/v3/projects/39584fb2ed764b26831eac9343a1d9c9/users/d7b269b260eb4a96aa4bc25ce7962e27/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:17.930000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2a4f7-db5a-420c-a9d8-0258cbe80df9 None admin] RBAC: Authorizing `identity:create_grant(project_id=39584fb2ed764b26831eac9343a1d9c9, user_id=d7b269b260eb4a96aa4bc25ce7962e27, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:17.931696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2a4f7-db5a-420c-a9d8-0258cbe80df9 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:17.947244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53f6c4-2b39-4849-9902-bfdd84fa2356 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:17.950500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 72/138] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 12:59:17 2022] PUT /identity/v3/projects/39584fb2ed764b26831eac9343a1d9c9/users/d7b269b260eb4a96aa4bc25ce7962e27/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 12:59:17.952087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2a4f7-db5a-420c-a9d8-0258cbe80df9 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:18.006140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281cb0e-d89a-45c3-9a7e-1c5a2942793f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:18.006861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281cb0e-d89a-45c3-9a7e-1c5a2942793f None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:18.007175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281cb0e-d89a-45c3-9a7e-1c5a2942793f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:18.007458 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281cb0e-d89a-45c3-9a7e-1c5a2942793f None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2/users/8bfb2d4384094e9e8a8b910061e5c9d4/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:18.009730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281cb0e-d89a-45c3-9a7e-1c5a2942793f None admin] RBAC: Authorizing `identity:create_grant(project_id=81be097728134d76957fdb5755382cc2, user_id=8bfb2d4384094e9e8a8b910061e5c9d4, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:18.011935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281cb0e-d89a-45c3-9a7e-1c5a2942793f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:18.032089 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 67/139] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 12:59:17 2022] PUT /identity/v3/projects/81be097728134d76957fdb5755382cc2/users/8bfb2d4384094e9e8a8b910061e5c9d4/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:59:18.033489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c150a-4bac-4d18-b38d-5aa2ff2e7023 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:18.033784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c150a-4bac-4d18-b38d-5aa2ff2e7023 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:18.034062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c150a-4bac-4d18-b38d-5aa2ff2e7023 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:18.043042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 68/140] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:18.050572 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74075baa-9543-474f-b15d-98b0e59aef97 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:18.051189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74075baa-9543-474f-b15d-98b0e59aef97 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:18.051720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74075baa-9543-474f-b15d-98b0e59aef97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:18.120427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-74075baa-9543-474f-b15d-98b0e59aef97 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:18.121684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-74075baa-9543-474f-b15d-98b0e59aef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:18.168594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:18.230961 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2a4f7-db5a-420c-a9d8-0258cbe80df9 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:18.283500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:18.294947 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 69/141] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:18.332155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c8ef6-87a0-4757-ab9f-a24c6b4c56a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:18.332862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c8ef6-87a0-4757-ab9f-a24c6b4c56a1 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:18.333160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c8ef6-87a0-4757-ab9f-a24c6b4c56a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:18.333446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c8ef6-87a0-4757-ab9f-a24c6b4c56a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:18.352220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c8ef6-87a0-4757-ab9f-a24c6b4c56a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-aa2c8ef6-87a0-4757-ab9f-a24c6b4c56a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:18.358063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:18.358993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:18.359810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:18.359810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:18.359810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:18.359810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:18.359810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:18.359810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:18.359810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:18.359810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:18.359810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:18.359810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:18.359810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:18.359810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:18.359810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 73/142] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:18.365451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74075baa-9543-474f-b15d-98b0e59aef97 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:18.538876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74075baa-9543-474f-b15d-98b0e59aef97 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:18.610627 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3dd40-191f-4ed9-b77f-a2ebc1468cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:18.614303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3dd40-191f-4ed9-b77f-a2ebc1468cdf None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:18.614790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3dd40-191f-4ed9-b77f-a2ebc1468cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:18.615159 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3dd40-191f-4ed9-b77f-a2ebc1468cdf None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:18.616278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3dd40-191f-4ed9-b77f-a2ebc1468cdf None admin] RBAC: Adding query filter params (name=member) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:18.616638 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3dd40-191f-4ed9-b77f-a2ebc1468cdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:18.619032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3dd40-191f-4ed9-b77f-a2ebc1468cdf None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:18.635023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 70/143] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:18.641266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c8ef6-87a0-4757-ab9f-a24c6b4c56a1 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:18.840552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c8ef6-87a0-4757-ab9f-a24c6b4c56a1 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:18.892795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9221045-5874-4202-823b-b49a23856243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:18.893359 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9221045-5874-4202-823b-b49a23856243 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:18.893641 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9221045-5874-4202-823b-b49a23856243 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:18.893925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9221045-5874-4202-823b-b49a23856243 None admin] PATH_INFO: `/v3/users/d7b269b260eb4a96aa4bc25ce7962e27` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:18.895191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9221045-5874-4202-823b-b49a23856243 None admin] RBAC: Authorizing `identity:get_user(user_id=d7b269b260eb4a96aa4bc25ce7962e27)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:18.896735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9221045-5874-4202-823b-b49a23856243 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:18.898426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 74/144] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:18 2022] GET /identity/v3/users/d7b269b260eb4a96aa4bc25ce7962e27 => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:18.904759 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3dd40-191f-4ed9-b77f-a2ebc1468cdf None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:18.916198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9221045-5874-4202-823b-b49a23856243 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:18.927614 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:19.046008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:19.171940 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3dd40-191f-4ed9-b77f-a2ebc1468cdf None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:19.221366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9221045-5874-4202-823b-b49a23856243 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:19.281897 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d64fb1-a255-4294-b8e8-7c7ffedbedc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:19.282548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d64fb1-a255-4294-b8e8-7c7ffedbedc3 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:19.282836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d64fb1-a255-4294-b8e8-7c7ffedbedc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:19.283131 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d64fb1-a255-4294-b8e8-7c7ffedbedc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:19.352258 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce13590-a1ef-4cb0-96a4-ae5d260d171a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:19.353231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce13590-a1ef-4cb0-96a4-ae5d260d171a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:19.353653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce13590-a1ef-4cb0-96a4-ae5d260d171a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:19.354067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce13590-a1ef-4cb0-96a4-ae5d260d171a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:19.356035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce13590-a1ef-4cb0-96a4-ae5d260d171a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:19.364934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce13590-a1ef-4cb0-96a4-ae5d260d171a None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:19.368285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 75/145] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:59:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:19.374663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce13590-a1ef-4cb0-96a4-ae5d260d171a None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:19.414070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d64fb1-a255-4294-b8e8-7c7ffedbedc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:19.432070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d64fb1-a255-4294-b8e8-7c7ffedbedc3 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:19.551489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:19.551683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:19.596719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce13590-a1ef-4cb0-96a4-ae5d260d171a None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:19.626074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 71/146] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:19.631837 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d64fb1-a255-4294-b8e8-7c7ffedbedc3 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:19.664799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b84af-8db0-4560-be57-3386da27e7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:19.665951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b84af-8db0-4560-be57-3386da27e7ab None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:19.666516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b84af-8db0-4560-be57-3386da27e7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:19.667005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b84af-8db0-4560-be57-3386da27e7ab None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:19.668277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b84af-8db0-4560-be57-3386da27e7ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:19.671220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b84af-8db0-4560-be57-3386da27e7ab None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:19.713456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 76/147] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 12:59:19 2022] POST /identity/v3/users => generated 283 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:59:19.791682 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d64fb1-a255-4294-b8e8-7c7ffedbedc3 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:19.840068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ea0cf-d7c2-4b01-ba7d-325a40adc4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:19.840769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ea0cf-d7c2-4b01-ba7d-325a40adc4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:19.841061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ea0cf-d7c2-4b01-ba7d-325a40adc4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:19.841342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ea0cf-d7c2-4b01-ba7d-325a40adc4f4 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:19.842796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ea0cf-d7c2-4b01-ba7d-325a40adc4f4 None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:19.844522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ea0cf-d7c2-4b01-ba7d-325a40adc4f4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:19.846383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 72/148] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:19 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:19.852837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b84af-8db0-4560-be57-3386da27e7ab None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:20.001608 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b84af-8db0-4560-be57-3386da27e7ab None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:20.032403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:20.041814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566d894-4625-4829-912c-b3307153ab52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:20.043006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566d894-4625-4829-912c-b3307153ab52 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:20.043631 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566d894-4625-4829-912c-b3307153ab52 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:20.044188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566d894-4625-4829-912c-b3307153ab52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:20.045862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566d894-4625-4829-912c-b3307153ab52 None admin] RBAC: Adding query filter params (role.id=86f185569d134d80841142ae8e9d3f47, scope.project.id=81be097728134d76957fdb5755382cc2, user.id=d7b269b260eb4a96aa4bc25ce7962e27) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:20.046481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566d894-4625-4829-912c-b3307153ab52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:20.048233 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566d894-4625-4829-912c-b3307153ab52 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:20.055439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 77/149] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:19 2022] GET /identity/v3/role_assignments?user.id=d7b269b260eb4a96aa4bc25ce7962e27&scope.project.id=81be097728134d76957fdb5755382cc2&role.id=86f185569d134d80841142ae8e9d3f47 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:20.628796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d295a-aa5d-4c00-9489-6a7c8644bfc1 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:20.629346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d295a-aa5d-4c00-9489-6a7c8644bfc1 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:20.629818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d295a-aa5d-4c00-9489-6a7c8644bfc1 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:20.632332 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 73/150] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:20.639649 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40523ff1-b9e7-4598-b4d5-39382a3301b4 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:20.640021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40523ff1-b9e7-4598-b4d5-39382a3301b4 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:20.640345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40523ff1-b9e7-4598-b4d5-39382a3301b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:20.675981 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-40523ff1-b9e7-4598-b4d5-39382a3301b4 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:20.677821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-40523ff1-b9e7-4598-b4d5-39382a3301b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:20.808477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 78/151] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:21.097032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d295a-aa5d-4c00-9489-6a7c8644bfc1 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:21.141194 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:21.254431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d295a-aa5d-4c00-9489-6a7c8644bfc1 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:21.303553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb1e19-5149-4a63-9b7d-5452d0e199da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:21.304356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb1e19-5149-4a63-9b7d-5452d0e199da None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:21.304680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb1e19-5149-4a63-9b7d-5452d0e199da None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:21.304955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb1e19-5149-4a63-9b7d-5452d0e199da None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:21.320066 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb1e19-5149-4a63-9b7d-5452d0e199da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-1bdb1e19-5149-4a63-9b7d-5452d0e199da None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:21.325308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:21.328542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:21.331699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:21.331699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:21.331699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:21.331699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:21.331699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:21.331699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:21.331699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:21.331699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:21.331699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:21.331699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:21.331699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:21.331699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:21.331699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 74/152] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:59:21.334392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40523ff1-b9e7-4598-b4d5-39382a3301b4 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:21.498134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40523ff1-b9e7-4598-b4d5-39382a3301b4 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:21.539716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:21.556520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7aaf46-9ec5-4b5b-a1f9-2c551e3a844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:21.557163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7aaf46-9ec5-4b5b-a1f9-2c551e3a844e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:21.557443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7aaf46-9ec5-4b5b-a1f9-2c551e3a844e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:21.557714 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7aaf46-9ec5-4b5b-a1f9-2c551e3a844e None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:21.558417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7aaf46-9ec5-4b5b-a1f9-2c551e3a844e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:21.558721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7aaf46-9ec5-4b5b-a1f9-2c551e3a844e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:21.560239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7aaf46-9ec5-4b5b-a1f9-2c551e3a844e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:21.574700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 79/153] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:21.580895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb1e19-5149-4a63-9b7d-5452d0e199da None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:21.681808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27948a-48f7-4e83-9bad-c321499867db None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:21.682563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27948a-48f7-4e83-9bad-c321499867db None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:21.682563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27948a-48f7-4e83-9bad-c321499867db None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:21.685327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 80/154] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:21.691328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eff0e-f186-4661-9712-0f9d7ac8e129 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:21.691715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eff0e-f186-4661-9712-0f9d7ac8e129 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:21.691949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eff0e-f186-4661-9712-0f9d7ac8e129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:21.719829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:21.722741 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ac0eff0e-f186-4661-9712-0f9d7ac8e129 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:21.724128 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ac0eff0e-f186-4661-9712-0f9d7ac8e129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:21.747374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb1e19-5149-4a63-9b7d-5452d0e199da None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:21.821440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0250e33-5708-41a5-bcb9-f26894ac50b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:21.822165 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0250e33-5708-41a5-bcb9-f26894ac50b4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:21.822551 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0250e33-5708-41a5-bcb9-f26894ac50b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:21.822983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0250e33-5708-41a5-bcb9-f26894ac50b4 None admin] PATH_INFO: `/v3/users/328674bcbff943fb878160494186b1b4` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:21.826628 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0250e33-5708-41a5-bcb9-f26894ac50b4 None admin] RBAC: Authorizing `identity:get_user(user_id=328674bcbff943fb878160494186b1b4)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:21.828164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0250e33-5708-41a5-bcb9-f26894ac50b4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:21.830320 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 75/155] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:21 2022] GET /identity/v3/users/328674bcbff943fb878160494186b1b4 => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:21.837170 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0250e33-5708-41a5-bcb9-f26894ac50b4 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:21.907067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 81/156] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:59:22.013537 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0250e33-5708-41a5-bcb9-f26894ac50b4 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:22.058356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226607d2-01cf-4408-8b7c-98d819cbb466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:22.059036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226607d2-01cf-4408-8b7c-98d819cbb466 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:22.059356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226607d2-01cf-4408-8b7c-98d819cbb466 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:22.059631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226607d2-01cf-4408-8b7c-98d819cbb466 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:22.061561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226607d2-01cf-4408-8b7c-98d819cbb466 None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:22.063146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226607d2-01cf-4408-8b7c-98d819cbb466 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:22.065163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 76/157] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:21 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:22.071354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0eff0e-f186-4661-9712-0f9d7ac8e129 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:22.166434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36eb03-5064-4fd6-8f24-2b92eb505cb0 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:22.166666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36eb03-5064-4fd6-8f24-2b92eb505cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:22.167012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36eb03-5064-4fd6-8f24-2b92eb505cb0 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:22.168613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 77/158] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:22.174151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e887d30-9743-44e9-8e76-8348d8f93005 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:22.174539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e887d30-9743-44e9-8e76-8348d8f93005 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:22.174808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e887d30-9743-44e9-8e76-8348d8f93005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:22.208890 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-6e887d30-9743-44e9-8e76-8348d8f93005 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:22.210124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-6e887d30-9743-44e9-8e76-8348d8f93005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:22.228831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0eff0e-f186-4661-9712-0f9d7ac8e129 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:22.244212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:22.285693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:22.346684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 78/159] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:59:22.353505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1ff69-04d5-4a41-bb2b-6b470c3a9646 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:22.353784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1ff69-04d5-4a41-bb2b-6b470c3a9646 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:22.354086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1ff69-04d5-4a41-bb2b-6b470c3a9646 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:22.356854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03f72b-41ad-4f08-afd2-a83cff45fa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:22.358259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f72b-41ad-4f08-afd2-a83cff45fa40 None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:22.358614 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f72b-41ad-4f08-afd2-a83cff45fa40 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:22.358936 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f72b-41ad-4f08-afd2-a83cff45fa40 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2/users/328674bcbff943fb878160494186b1b4/roles/4b22e77ca0b449f3b83e08e32d811efb` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:22.360226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 79/160] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:22 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:22.365946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03f72b-41ad-4f08-afd2-a83cff45fa40 None admin] RBAC: Authorizing `identity:create_grant(project_id=81be097728134d76957fdb5755382cc2, user_id=328674bcbff943fb878160494186b1b4, role_id=4b22e77ca0b449f3b83e08e32d811efb)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:22.367620 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03f72b-41ad-4f08-afd2-a83cff45fa40 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:22.371435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7d97c-726b-4f08-9b0a-57ca08a1a5d9 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:22.371813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7d97c-726b-4f08-9b0a-57ca08a1a5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:22.371963 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7d97c-726b-4f08-9b0a-57ca08a1a5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:22.395830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 82/161] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 12:59:22 2022] PUT /identity/v3/projects/81be097728134d76957fdb5755382cc2/users/328674bcbff943fb878160494186b1b4/roles/4b22e77ca0b449f3b83e08e32d811efb => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:59:22.410204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-d6a7d97c-726b-4f08-9b0a-57ca08a1a5d9 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:22.411520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d6a7d97c-726b-4f08-9b0a-57ca08a1a5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:22.467822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03f72b-41ad-4f08-afd2-a83cff45fa40 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:22.573901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 80/162] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:22.688895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7d97c-726b-4f08-9b0a-57ca08a1a5d9 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:22.697257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03f72b-41ad-4f08-afd2-a83cff45fa40 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:22.757576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c37c49-a6e6-458f-ac3d-d209f9e377ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:22.758159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c37c49-a6e6-458f-ac3d-d209f9e377ed None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:22.758441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c37c49-a6e6-458f-ac3d-d209f9e377ed None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:22.758709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c37c49-a6e6-458f-ac3d-d209f9e377ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:22.783531 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c37c49-a6e6-458f-ac3d-d209f9e377ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-84c37c49-a6e6-458f-ac3d-d209f9e377ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:22.790050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:22.791028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:22.791951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:22.791951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:22.791951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:22.791951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:22.791951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:22.791951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:22.791951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:22.791951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:22.791951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:22.791951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:22.791951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:22.791951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:22.791951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 83/163] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:22.799831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c37c49-a6e6-458f-ac3d-d209f9e377ed None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:22.861409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:22.939537 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:22.960937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c37c49-a6e6-458f-ac3d-d209f9e377ed None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:22.985480 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7d97c-726b-4f08-9b0a-57ca08a1a5d9 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:23.012987 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545e0ae-f4b5-4882-a121-88df6f429cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:23.014098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545e0ae-f4b5-4882-a121-88df6f429cfb None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:23.014377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545e0ae-f4b5-4882-a121-88df6f429cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:23.014649 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545e0ae-f4b5-4882-a121-88df6f429cfb None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:23.015383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545e0ae-f4b5-4882-a121-88df6f429cfb None admin] RBAC: Adding query filter params (name=member) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:23.015690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545e0ae-f4b5-4882-a121-88df6f429cfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:23.017155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545e0ae-f4b5-4882-a121-88df6f429cfb None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:23.048513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 84/164] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:23.049211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551408fd-9d7e-4195-bc02-bb9aa336fc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:23.049935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551408fd-9d7e-4195-bc02-bb9aa336fc70 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:23.050217 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551408fd-9d7e-4195-bc02-bb9aa336fc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:23.050493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551408fd-9d7e-4195-bc02-bb9aa336fc70 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:23.054725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545e0ae-f4b5-4882-a121-88df6f429cfb None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:23.092171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551408fd-9d7e-4195-bc02-bb9aa336fc70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-551408fd-9d7e-4195-bc02-bb9aa336fc70 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:23.100482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:23.101441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:23.102219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:23.102219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:23.102219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:23.102219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:23.102219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:23.102219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:23.102219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:23.102219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:23.102219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:23.102219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:23.102219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:23.102219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:23.111249 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 81/165] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 12:59:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:59:23.144685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551408fd-9d7e-4195-bc02-bb9aa336fc70 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:23.287603 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:23.296468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545e0ae-f4b5-4882-a121-88df6f429cfb None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:23.405255 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca9917-04bf-4d62-899f-7d29271acb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:23.406451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca9917-04bf-4d62-899f-7d29271acb03 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:23.407375 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca9917-04bf-4d62-899f-7d29271acb03 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:23.407658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca9917-04bf-4d62-899f-7d29271acb03 None admin] PATH_INFO: `/v3/users/d7b269b260eb4a96aa4bc25ce7962e27` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:23.409104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca9917-04bf-4d62-899f-7d29271acb03 None admin] RBAC: Authorizing `identity:get_user(user_id=d7b269b260eb4a96aa4bc25ce7962e27)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:23.411606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca9917-04bf-4d62-899f-7d29271acb03 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:23.416380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551408fd-9d7e-4195-bc02-bb9aa336fc70 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:23.423707 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 85/166] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:23 2022] GET /identity/v3/users/d7b269b260eb4a96aa4bc25ce7962e27 => generated 275 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:23.424793 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca9917-04bf-4d62-899f-7d29271acb03 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:23.498380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e5eab-6f76-4420-9bc9-85af45b5989f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:23.499083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e5eab-6f76-4420-9bc9-85af45b5989f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:23.499391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e5eab-6f76-4420-9bc9-85af45b5989f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:23.499660 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e5eab-6f76-4420-9bc9-85af45b5989f None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:23.500420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e5eab-6f76-4420-9bc9-85af45b5989f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:23.500728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e5eab-6f76-4420-9bc9-85af45b5989f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:23.502136 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e5eab-6f76-4420-9bc9-85af45b5989f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:23.533185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 82/167] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 12:59:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:23.534212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e5eab-6f76-4420-9bc9-85af45b5989f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:23.553939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:23.689830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca9917-04bf-4d62-899f-7d29271acb03 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:23.734910 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e5eab-6f76-4420-9bc9-85af45b5989f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:23.755990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9cbcf-46f3-4923-bd9f-7c8f03419766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:23.756700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9cbcf-46f3-4923-bd9f-7c8f03419766 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:23.757111 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9cbcf-46f3-4923-bd9f-7c8f03419766 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:23.757213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9cbcf-46f3-4923-bd9f-7c8f03419766 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:23.777412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9cbcf-46f3-4923-bd9f-7c8f03419766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-3ac9cbcf-46f3-4923-bd9f-7c8f03419766 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:23.783419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:23.784842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 86/168] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 12:59:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:23.785938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9cbcf-46f3-4923-bd9f-7c8f03419766 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:23.794117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ad28c-7a16-4aff-b1d1-ed41542eaf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:23.795779 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ad28c-7a16-4aff-b1d1-ed41542eaf46 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:23.796304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ad28c-7a16-4aff-b1d1-ed41542eaf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:23.796773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ad28c-7a16-4aff-b1d1-ed41542eaf46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:23.817990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:23.899463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ad28c-7a16-4aff-b1d1-ed41542eaf46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:23.901340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ad28c-7a16-4aff-b1d1-ed41542eaf46 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:23.907639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9cbcf-46f3-4923-bd9f-7c8f03419766 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:23.911341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:23.959992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfeb36f-c99c-4d12-ae4b-4ab639b03e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:23.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfeb36f-c99c-4d12-ae4b-4ab639b03e1b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:23.961429 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfeb36f-c99c-4d12-ae4b-4ab639b03e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:23.961868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfeb36f-c99c-4d12-ae4b-4ab639b03e1b None admin] PATH_INFO: `/v3/users/d7b269b260eb4a96aa4bc25ce7962e27` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:23.963423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfeb36f-c99c-4d12-ae4b-4ab639b03e1b None admin] RBAC: Authorizing `identity:get_user(user_id=d7b269b260eb4a96aa4bc25ce7962e27)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:23.965121 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfeb36f-c99c-4d12-ae4b-4ab639b03e1b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:23.971254 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 87/169] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:23 2022] GET /identity/v3/users/d7b269b260eb4a96aa4bc25ce7962e27 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:23.972498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfeb36f-c99c-4d12-ae4b-4ab639b03e1b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:24.043143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 83/170] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:24.043143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ad28c-7a16-4aff-b1d1-ed41542eaf46 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:24.141807 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:24.164039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfeb36f-c99c-4d12-ae4b-4ab639b03e1b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:24.236000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033a04f-2f3c-444a-8f16-14c82b7b4a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:24.236645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033a04f-2f3c-444a-8f16-14c82b7b4a87 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:24.236926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033a04f-2f3c-444a-8f16-14c82b7b4a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:24.237192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033a04f-2f3c-444a-8f16-14c82b7b4a87 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:24.237901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033a04f-2f3c-444a-8f16-14c82b7b4a87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:24.238207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033a04f-2f3c-444a-8f16-14c82b7b4a87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:24.239682 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033a04f-2f3c-444a-8f16-14c82b7b4a87 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:24.254582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 88/171] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:24.255511 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033a04f-2f3c-444a-8f16-14c82b7b4a87 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:24.297574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ad28c-7a16-4aff-b1d1-ed41542eaf46 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:24.311876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:24.398882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c61a6f-6632-47fc-ac7c-433faf9fa66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:24.399482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c61a6f-6632-47fc-ac7c-433faf9fa66f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:24.399769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c61a6f-6632-47fc-ac7c-433faf9fa66f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:24.400049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c61a6f-6632-47fc-ac7c-433faf9fa66f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:24.494499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033a04f-2f3c-444a-8f16-14c82b7b4a87 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:24.585920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4aaaf6-1dd2-4b86-a302-a5853a6baa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:24.586638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aaaf6-1dd2-4b86-a302-a5853a6baa26 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:24.586920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aaaf6-1dd2-4b86-a302-a5853a6baa26 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:24.587504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aaaf6-1dd2-4b86-a302-a5853a6baa26 None admin] PATH_INFO: `/v3/projects/39584fb2ed764b26831eac9343a1d9c9` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:24.588724 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4aaaf6-1dd2-4b86-a302-a5853a6baa26 None admin] RBAC: Authorizing `identity:get_project(project_id=39584fb2ed764b26831eac9343a1d9c9)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:24.590794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4aaaf6-1dd2-4b86-a302-a5853a6baa26 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:24.593801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 89/172] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:24 2022] GET /identity/v3/projects/39584fb2ed764b26831eac9343a1d9c9 => generated 312 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:24.594967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4aaaf6-1dd2-4b86-a302-a5853a6baa26 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:24.627608 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c61a6f-6632-47fc-ac7c-433faf9fa66f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:24.631212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c61a6f-6632-47fc-ac7c-433faf9fa66f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:24.687756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:24.813117 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4aaaf6-1dd2-4b86-a302-a5853a6baa26 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:24.853745 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 84/173] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 817 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:24.854731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c61a6f-6632-47fc-ac7c-433faf9fa66f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:24.868664 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:24.886113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6001f35-9d87-490f-b9c1-2ba04a7af437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:24.886779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6001f35-9d87-490f-b9c1-2ba04a7af437 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:24.887053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6001f35-9d87-490f-b9c1-2ba04a7af437 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:24.887366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6001f35-9d87-490f-b9c1-2ba04a7af437 None admin] PATH_INFO: `/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:24.888948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6001f35-9d87-490f-b9c1-2ba04a7af437 None admin] RBAC: Authorizing `identity:get_user(user_id=8bfb2d4384094e9e8a8b910061e5c9d4)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:24.890914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6001f35-9d87-490f-b9c1-2ba04a7af437 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:24.892918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 90/174] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:24 2022] GET /identity/v3/users/8bfb2d4384094e9e8a8b910061e5c9d4 => generated 247 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:24.894153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3ddbb-e75b-4089-b770-567c90e7176e None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:24.894419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3ddbb-e75b-4089-b770-567c90e7176e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:24.894687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3ddbb-e75b-4089-b770-567c90e7176e None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:24.896244 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 91/175] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:24.896835 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3ddbb-e75b-4089-b770-567c90e7176e None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:25.037379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c61a6f-6632-47fc-ac7c-433faf9fa66f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:25.085556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4be1e0-77ab-4157-b024-efaae5b0394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:25.085556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4be1e0-77ab-4157-b024-efaae5b0394f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:25.085556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4be1e0-77ab-4157-b024-efaae5b0394f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:25.085556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4be1e0-77ab-4157-b024-efaae5b0394f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:25.086757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4be1e0-77ab-4157-b024-efaae5b0394f None admin] RBAC: Adding query filter params (role.id=86f185569d134d80841142ae8e9d3f47, scope.project.id=39584fb2ed764b26831eac9343a1d9c9, user.id=d7b269b260eb4a96aa4bc25ce7962e27) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:25.086757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4be1e0-77ab-4157-b024-efaae5b0394f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:25.086757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4be1e0-77ab-4157-b024-efaae5b0394f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:25.094532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 85/176] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:24 2022] GET /identity/v3/role_assignments?user.id=d7b269b260eb4a96aa4bc25ce7962e27&scope.project.id=39584fb2ed764b26831eac9343a1d9c9&role.id=86f185569d134d80841142ae8e9d3f47 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:25.095889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132f300-0628-49c5-8646-a97a684f70a7 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:25.096189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132f300-0628-49c5-8646-a97a684f70a7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:25.096467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132f300-0628-49c5-8646-a97a684f70a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:25.150864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-c132f300-0628-49c5-8646-a97a684f70a7 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:25.152132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c132f300-0628-49c5-8646-a97a684f70a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:25.193595 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:25.229029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3ddbb-e75b-4089-b770-567c90e7176e None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:25.327139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b3860-e50c-45bb-89bc-806409ce1068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:25.327139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b3860-e50c-45bb-89bc-806409ce1068 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:25.327139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b3860-e50c-45bb-89bc-806409ce1068 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:25.327139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b3860-e50c-45bb-89bc-806409ce1068 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:25.327813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b3860-e50c-45bb-89bc-806409ce1068 None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:25.327813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b3860-e50c-45bb-89bc-806409ce1068 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:25.327813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 92/177] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:24 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:25.328805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b3860-e50c-45bb-89bc-806409ce1068 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:25.393900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 86/178] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:25.394808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132f300-0628-49c5-8646-a97a684f70a7 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:25.398406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:25.615401 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b3860-e50c-45bb-89bc-806409ce1068 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:25.638996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132f300-0628-49c5-8646-a97a684f70a7 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:25.701801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7d528-fc09-40b3-91b2-37acdf3f5134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:25.702558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7d528-fc09-40b3-91b2-37acdf3f5134 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:25.702837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7d528-fc09-40b3-91b2-37acdf3f5134 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:25.703142 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7d528-fc09-40b3-91b2-37acdf3f5134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:25.711094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d748e-bf83-41d7-b3b9-778cce7c5775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:25.712181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d748e-bf83-41d7-b3b9-778cce7c5775 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:25.712639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d748e-bf83-41d7-b3b9-778cce7c5775 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:25.713096 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d748e-bf83-41d7-b3b9-778cce7c5775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:25.714480 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d748e-bf83-41d7-b3b9-778cce7c5775 None admin] RBAC: Adding query filter params (role.id=649721c72e5d40deabd3512c529ffdf4, scope.project.id=81be097728134d76957fdb5755382cc2, user.id=d7b269b260eb4a96aa4bc25ce7962e27) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:25.715038 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d748e-bf83-41d7-b3b9-778cce7c5775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:25.717651 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d748e-bf83-41d7-b3b9-778cce7c5775 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:25.729120 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 87/179] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:25 2022] GET /identity/v3/role_assignments?user.id=d7b269b260eb4a96aa4bc25ce7962e27&scope.project.id=81be097728134d76957fdb5755382cc2&role.id=649721c72e5d40deabd3512c529ffdf4 => generated 608 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:25.734813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c210f0-ff16-4411-a553-c5c52dc67470 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:25.735564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c210f0-ff16-4411-a553-c5c52dc67470 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:25.736082 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c210f0-ff16-4411-a553-c5c52dc67470 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:25.738651 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 88/180] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:25.752369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0df40f-732d-4edf-a6a8-74dde1344483 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:25.752874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0df40f-732d-4edf-a6a8-74dde1344483 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:25.753355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0df40f-732d-4edf-a6a8-74dde1344483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:25.764155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:25.796336 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-bf0df40f-732d-4edf-a6a8-74dde1344483 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:25.797582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bf0df40f-732d-4edf-a6a8-74dde1344483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:25.871751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7d528-fc09-40b3-91b2-37acdf3f5134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:25.874973 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7d528-fc09-40b3-91b2-37acdf3f5134 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:25.953425 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:25.964514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 89/181] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:59:25.965912 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97c292-fac3-40e3-a7f4-811a8482a4da None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:25.966206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97c292-fac3-40e3-a7f4-811a8482a4da None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:25.966521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97c292-fac3-40e3-a7f4-811a8482a4da None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:25.968461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 90/182] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:25.969368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97c292-fac3-40e3-a7f4-811a8482a4da None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:26.071245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 93/183] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:26.072963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35041c2-051a-4537-be8d-9b22a6bc243d None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:26.073185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35041c2-051a-4537-be8d-9b22a6bc243d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:26.073463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35041c2-051a-4537-be8d-9b22a6bc243d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:26.098678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-b35041c2-051a-4537-be8d-9b22a6bc243d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:26.098678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b35041c2-051a-4537-be8d-9b22a6bc243d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:26.145611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97c292-fac3-40e3-a7f4-811a8482a4da None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:26.186618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 94/184] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:26.187453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35041c2-051a-4537-be8d-9b22a6bc243d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:26.193478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64c06b-40a0-4994-8a82-f7e73fbcf209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:26.194415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64c06b-40a0-4994-8a82-f7e73fbcf209 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:26.194840 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64c06b-40a0-4994-8a82-f7e73fbcf209 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:26.195287 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64c06b-40a0-4994-8a82-f7e73fbcf209 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:26.209333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64c06b-40a0-4994-8a82-f7e73fbcf209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-1f64c06b-40a0-4994-8a82-f7e73fbcf209 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:26.214949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:26.216025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:26.217138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:26.217138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:26.217138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:26.217138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:26.217138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:26.217138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:26.217138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:26.217138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:26.217138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:26.217138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:26.217138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:59:26.217138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:26.219934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 91/185] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 12:59:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:26.226677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64c06b-40a0-4994-8a82-f7e73fbcf209 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:26.260809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:26.301441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:26.352719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35041c2-051a-4537-be8d-9b22a6bc243d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:26.362733 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64c06b-40a0-4994-8a82-f7e73fbcf209 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:26.402753 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785f67b-bfa6-41f7-8faa-ba06b8479b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:26.403363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785f67b-bfa6-41f7-8faa-ba06b8479b87 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:26.403646 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785f67b-bfa6-41f7-8faa-ba06b8479b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:26.403978 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785f67b-bfa6-41f7-8faa-ba06b8479b87 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:26.404844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4a785-fe31-4d1d-b904-6ea87bac1fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:26.405198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785f67b-bfa6-41f7-8faa-ba06b8479b87 None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:26.405988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4a785-fe31-4d1d-b904-6ea87bac1fde None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:26.406501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4a785-fe31-4d1d-b904-6ea87bac1fde None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:26.406842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785f67b-bfa6-41f7-8faa-ba06b8479b87 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:26.407215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4a785-fe31-4d1d-b904-6ea87bac1fde None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:26.408231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4a785-fe31-4d1d-b904-6ea87bac1fde None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:26.408652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 95/186] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:26 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:26.409193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4a785-fe31-4d1d-b904-6ea87bac1fde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:26.409853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785f67b-bfa6-41f7-8faa-ba06b8479b87 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:26.411154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4a785-fe31-4d1d-b904-6ea87bac1fde None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:26.424176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 92/187] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:26.430319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4a785-fe31-4d1d-b904-6ea87bac1fde None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:26.483642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:26.537062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785f67b-bfa6-41f7-8faa-ba06b8479b87 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:26.556758 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4a785-fe31-4d1d-b904-6ea87bac1fde None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:26.577271 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafae3f-bb85-4705-ad2c-79505cc3c503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:26.577937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafae3f-bb85-4705-ad2c-79505cc3c503 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:26.578260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafae3f-bb85-4705-ad2c-79505cc3c503 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:26.578727 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafae3f-bb85-4705-ad2c-79505cc3c503 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:26.580055 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafae3f-bb85-4705-ad2c-79505cc3c503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:26.581626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafae3f-bb85-4705-ad2c-79505cc3c503 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:26.583509 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 96/188] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:59:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:26.584423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafae3f-bb85-4705-ad2c-79505cc3c503 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:26.591845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:26.615228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07180599-c924-49fb-bca4-c15e72acd6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:26.615855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07180599-c924-49fb-bca4-c15e72acd6bf None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:26.616162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07180599-c924-49fb-bca4-c15e72acd6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:26.616439 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07180599-c924-49fb-bca4-c15e72acd6bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:26.617649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07180599-c924-49fb-bca4-c15e72acd6bf None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=81be097728134d76957fdb5755382cc2, user.id=8bfb2d4384094e9e8a8b910061e5c9d4) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:26.617649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07180599-c924-49fb-bca4-c15e72acd6bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:26.618972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07180599-c924-49fb-bca4-c15e72acd6bf None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:26.627893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 93/189] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:26 2022] GET /identity/v3/role_assignments?user.id=8bfb2d4384094e9e8a8b910061e5c9d4&scope.project.id=81be097728134d76957fdb5755382cc2&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:26.630531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679648f1-3e7c-47b5-881c-bd9249b36dc5 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:26.631609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679648f1-3e7c-47b5-881c-bd9249b36dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:26.632342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679648f1-3e7c-47b5-881c-bd9249b36dc5 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:26.636261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 94/190] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:26.637716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679648f1-3e7c-47b5-881c-bd9249b36dc5 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:26.773390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafae3f-bb85-4705-ad2c-79505cc3c503 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:26.814422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:26.828389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679648f1-3e7c-47b5-881c-bd9249b36dc5 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:26.834451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2ed23-9118-498f-b209-807a9097224d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:26.835356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2ed23-9118-498f-b209-807a9097224d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:26.835658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2ed23-9118-498f-b209-807a9097224d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:26.835999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2ed23-9118-498f-b209-807a9097224d None admin] PATH_INFO: `/v3/users/1dd14697d94246a284b0106fcc1b4f20` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:26.858156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2ed23-9118-498f-b209-807a9097224d None admin] RBAC: Authorizing `identity:get_user(user_id=1dd14697d94246a284b0106fcc1b4f20)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:26.860005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2ed23-9118-498f-b209-807a9097224d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:26.861779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 97/191] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:26 2022] GET /identity/v3/users/1dd14697d94246a284b0106fcc1b4f20 => generated 283 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:26.863207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342be0f2-7548-4b5d-b372-985f9ec921cc None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:26.863531 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342be0f2-7548-4b5d-b372-985f9ec921cc None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:26.863785 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342be0f2-7548-4b5d-b372-985f9ec921cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:26.887956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-342be0f2-7548-4b5d-b372-985f9ec921cc None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:26.890146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-342be0f2-7548-4b5d-b372-985f9ec921cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:26.904418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0035457-a130-4d1a-8d1c-b6bcf3ffb3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:26.905510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0035457-a130-4d1a-8d1c-b6bcf3ffb3db None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:26.906007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0035457-a130-4d1a-8d1c-b6bcf3ffb3db None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:26.906502 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0035457-a130-4d1a-8d1c-b6bcf3ffb3db None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:26.907603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0035457-a130-4d1a-8d1c-b6bcf3ffb3db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:26.910426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0035457-a130-4d1a-8d1c-b6bcf3ffb3db None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:26.952262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 95/192] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 12:59:26 2022] POST /identity/v3/users => generated 297 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:59:26.953828 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0035457-a130-4d1a-8d1c-b6bcf3ffb3db None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:27.025305 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:27.030854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 98/193] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:27.045474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342be0f2-7548-4b5d-b372-985f9ec921cc None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:27.172513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0035457-a130-4d1a-8d1c-b6bcf3ffb3db None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:27.227138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342be0f2-7548-4b5d-b372-985f9ec921cc None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:27.243968 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e259f9-c8e3-4cfb-8f9c-5d8b9478026d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:27.244630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e259f9-c8e3-4cfb-8f9c-5d8b9478026d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:27.244921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e259f9-c8e3-4cfb-8f9c-5d8b9478026d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:27.245238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e259f9-c8e3-4cfb-8f9c-5d8b9478026d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:27.255954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:27.261463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e259f9-c8e3-4cfb-8f9c-5d8b9478026d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-e9e259f9-c8e3-4cfb-8f9c-5d8b9478026d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:27.269803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:27.270875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:27.272037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:27.272037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:27.272037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:27.272037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:27.272037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:27.272037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:27.272037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:27.272037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:27.272037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:27.272037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:27.272037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:27.272037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:27.275052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 96/194] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 12:59:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:27.285527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e259f9-c8e3-4cfb-8f9c-5d8b9478026d None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:27.298482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529f1f2-4928-4245-b43f-3dda0efed260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:27.299626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529f1f2-4928-4245-b43f-3dda0efed260 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:27.300097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529f1f2-4928-4245-b43f-3dda0efed260 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:27.300651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529f1f2-4928-4245-b43f-3dda0efed260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:27.429953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529f1f2-4928-4245-b43f-3dda0efed260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:27.431499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529f1f2-4928-4245-b43f-3dda0efed260 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:27.450397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:27.458231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e259f9-c8e3-4cfb-8f9c-5d8b9478026d None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:27.509808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb6634-bd59-4cf7-97f4-b10bd928e9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:27.510441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb6634-bd59-4cf7-97f4-b10bd928e9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:27.510723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb6634-bd59-4cf7-97f4-b10bd928e9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:27.510996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb6634-bd59-4cf7-97f4-b10bd928e9b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:27.511736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb6634-bd59-4cf7-97f4-b10bd928e9b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:27.512040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb6634-bd59-4cf7-97f4-b10bd928e9b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:27.514521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb6634-bd59-4cf7-97f4-b10bd928e9b6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:27.524963 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 97/195] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 12:59:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:27.529818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb6634-bd59-4cf7-97f4-b10bd928e9b6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:27.573858 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 99/196] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:27.575117 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529f1f2-4928-4245-b43f-3dda0efed260 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:27.592379 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:27.653831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb6634-bd59-4cf7-97f4-b10bd928e9b6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:27.695172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529f1f2-4928-4245-b43f-3dda0efed260 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:27.695762 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f2437-f3f9-48bf-8135-5cd540776d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:27.696420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f2437-f3f9-48bf-8135-5cd540776d0d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:27.696775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f2437-f3f9-48bf-8135-5cd540776d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:27.697046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f2437-f3f9-48bf-8135-5cd540776d0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:27.714181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f2437-f3f9-48bf-8135-5cd540776d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-ab0f2437-f3f9-48bf-8135-5cd540776d0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:27.722281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:27.723454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:27.724240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:27.724240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:27.724240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:27.724240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:27.724240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:27.724240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:27.724240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:27.724240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:27.724240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:27.724240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:27.724240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:27.724240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:27.724240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 98/197] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:27.724908 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f2437-f3f9-48bf-8135-5cd540776d0d None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:27.741567 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e41f25-181d-48f6-af1c-8de196b3836b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:27.742633 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e41f25-181d-48f6-af1c-8de196b3836b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:27.743151 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e41f25-181d-48f6-af1c-8de196b3836b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:27.743651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e41f25-181d-48f6-af1c-8de196b3836b None admin] PATH_INFO: `/v3/users/1dd14697d94246a284b0106fcc1b4f20` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:27.745160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e41f25-181d-48f6-af1c-8de196b3836b None admin] RBAC: Authorizing `identity:get_user(user_id=1dd14697d94246a284b0106fcc1b4f20)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:27.746945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e41f25-181d-48f6-af1c-8de196b3836b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:27.753876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:27.754637 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 100/198] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:27 2022] GET /identity/v3/users/1dd14697d94246a284b0106fcc1b4f20 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:27.756141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e41f25-181d-48f6-af1c-8de196b3836b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:27.864400 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:27.890726 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e41f25-181d-48f6-af1c-8de196b3836b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:27.924510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f2437-f3f9-48bf-8135-5cd540776d0d None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:27.934357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774c38e-d739-49da-8417-0b92776fee6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:27.935966 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774c38e-d739-49da-8417-0b92776fee6a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:27.936491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774c38e-d739-49da-8417-0b92776fee6a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:27.936959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774c38e-d739-49da-8417-0b92776fee6a None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:27.937856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774c38e-d739-49da-8417-0b92776fee6a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:27.938376 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774c38e-d739-49da-8417-0b92776fee6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:27.940171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774c38e-d739-49da-8417-0b92776fee6a None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:27.951008 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 101/199] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:27.952358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774c38e-d739-49da-8417-0b92776fee6a None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:28.001662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9f3ee-238e-4893-b199-66493d6b9693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:28.002877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9f3ee-238e-4893-b199-66493d6b9693 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:28.003695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9f3ee-238e-4893-b199-66493d6b9693 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:28.004481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9f3ee-238e-4893-b199-66493d6b9693 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:28.006896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9f3ee-238e-4893-b199-66493d6b9693 None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:28.010302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9f3ee-238e-4893-b199-66493d6b9693 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:28.015666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 99/200] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:27 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:28.017191 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9f3ee-238e-4893-b199-66493d6b9693 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:28.097533 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774c38e-d739-49da-8417-0b92776fee6a None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:28.142283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff1099-fc02-4f0d-b1a8-ca25755d6129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:28.143502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff1099-fc02-4f0d-b1a8-ca25755d6129 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:28.144022 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff1099-fc02-4f0d-b1a8-ca25755d6129 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:28.144524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff1099-fc02-4f0d-b1a8-ca25755d6129 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:28.162198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:28.183612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:28.211971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9f3ee-238e-4893-b199-66493d6b9693 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:28.233536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff1099-fc02-4f0d-b1a8-ca25755d6129 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:28.235586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff1099-fc02-4f0d-b1a8-ca25755d6129 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:28.285288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06939c8-e08e-4fa6-83eb-8df21e0fd6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:28.286284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06939c8-e08e-4fa6-83eb-8df21e0fd6de None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:28.286693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06939c8-e08e-4fa6-83eb-8df21e0fd6de None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:28.287247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06939c8-e08e-4fa6-83eb-8df21e0fd6de None admin] PATH_INFO: `/v3/users/328674bcbff943fb878160494186b1b4` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:28.289040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06939c8-e08e-4fa6-83eb-8df21e0fd6de None admin] RBAC: Authorizing `identity:get_user(user_id=328674bcbff943fb878160494186b1b4)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:28.291417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06939c8-e08e-4fa6-83eb-8df21e0fd6de None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:28.295028 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 100/201] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:28 2022] GET /identity/v3/users/328674bcbff943fb878160494186b1b4 => generated 289 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:28.296774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06939c8-e08e-4fa6-83eb-8df21e0fd6de None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:28.366712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 102/202] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:28.368249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff1099-fc02-4f0d-b1a8-ca25755d6129 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:28.421084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:28.483362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06939c8-e08e-4fa6-83eb-8df21e0fd6de None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:28.491694 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff1099-fc02-4f0d-b1a8-ca25755d6129 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:28.533237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdda1d-3f14-4047-8533-92cffc07cb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:28.534335 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdda1d-3f14-4047-8533-92cffc07cb6a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:28.534837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdda1d-3f14-4047-8533-92cffc07cb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:28.535361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdda1d-3f14-4047-8533-92cffc07cb6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:28.555095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84a80a-d16c-4a01-b86f-9382d8847066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:28.556127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84a80a-d16c-4a01-b86f-9382d8847066 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:28.556630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84a80a-d16c-4a01-b86f-9382d8847066 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:28.557192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84a80a-d16c-4a01-b86f-9382d8847066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:28.559003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84a80a-d16c-4a01-b86f-9382d8847066 None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=717d2ed44c684799a826e03b360f281b, user.id=1dd14697d94246a284b0106fcc1b4f20) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:28.559699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84a80a-d16c-4a01-b86f-9382d8847066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:28.563444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84a80a-d16c-4a01-b86f-9382d8847066 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:28.574781 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 101/203] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:28 2022] GET /identity/v3/role_assignments?user.id=1dd14697d94246a284b0106fcc1b4f20&scope.project.id=717d2ed44c684799a826e03b360f281b&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:28.576505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84a80a-d16c-4a01-b86f-9382d8847066 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:28.624396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdda1d-3f14-4047-8533-92cffc07cb6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:28.626350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdda1d-3f14-4047-8533-92cffc07cb6a None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:28.643874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:28.767261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:28.833979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84a80a-d16c-4a01-b86f-9382d8847066 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:28.879362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee98c00-801e-40e4-8a95-f1bf4601e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:28.880496 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee98c00-801e-40e4-8a95-f1bf4601e21d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:28.880980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee98c00-801e-40e4-8a95-f1bf4601e21d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:28.881451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee98c00-801e-40e4-8a95-f1bf4601e21d None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:28.882952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee98c00-801e-40e4-8a95-f1bf4601e21d None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:28.884765 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee98c00-801e-40e4-8a95-f1bf4601e21d None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:28.887675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 102/204] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:28 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:28.897329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee98c00-801e-40e4-8a95-f1bf4601e21d None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:28.939631 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 103/205] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:28.952472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdda1d-3f14-4047-8533-92cffc07cb6a None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:29.035926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:29.056518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee98c00-801e-40e4-8a95-f1bf4601e21d None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:29.098537 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1ae6b-9bda-4365-af1d-462242c775b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:29.099185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1ae6b-9bda-4365-af1d-462242c775b0 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:29.099495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1ae6b-9bda-4365-af1d-462242c775b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:29.099736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1ae6b-9bda-4365-af1d-462242c775b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:29.100474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1ae6b-9bda-4365-af1d-462242c775b0 None admin] RBAC: Adding query filter params (role.id=649721c72e5d40deabd3512c529ffdf4, scope.project.id=717d2ed44c684799a826e03b360f281b, user.id=1dd14697d94246a284b0106fcc1b4f20) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:29.100783 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1ae6b-9bda-4365-af1d-462242c775b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:29.102843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1ae6b-9bda-4365-af1d-462242c775b0 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:29.111591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 103/206] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:28 2022] GET /identity/v3/role_assignments?user.id=1dd14697d94246a284b0106fcc1b4f20&scope.project.id=717d2ed44c684799a826e03b360f281b&role.id=649721c72e5d40deabd3512c529ffdf4 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:29.243796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdda1d-3f14-4047-8533-92cffc07cb6a None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:29.324988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617c9bf-0a97-4daf-950f-f8435ba708bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:29.326525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617c9bf-0a97-4daf-950f-f8435ba708bf None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:29.327271 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617c9bf-0a97-4daf-950f-f8435ba708bf None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:29.327993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617c9bf-0a97-4daf-950f-f8435ba708bf None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:29.332100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617c9bf-0a97-4daf-950f-f8435ba708bf None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:29.333918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617c9bf-0a97-4daf-950f-f8435ba708bf None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:29.335837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 104/207] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:28 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:29.345678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1ae6b-9bda-4365-af1d-462242c775b0 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:29.512102 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1ae6b-9bda-4365-af1d-462242c775b0 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:29.584259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051800e7-1a47-4fe7-98b3-84e97b49c79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:29.584811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051800e7-1a47-4fe7-98b3-84e97b49c79e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:29.585099 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051800e7-1a47-4fe7-98b3-84e97b49c79e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:29.587154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051800e7-1a47-4fe7-98b3-84e97b49c79e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:29.587154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051800e7-1a47-4fe7-98b3-84e97b49c79e None admin] RBAC: Adding query filter params (role.id=4b22e77ca0b449f3b83e08e32d811efb, scope.project.id=81be097728134d76957fdb5755382cc2, user.id=328674bcbff943fb878160494186b1b4) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:29.587154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051800e7-1a47-4fe7-98b3-84e97b49c79e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:29.588312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051800e7-1a47-4fe7-98b3-84e97b49c79e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:29.601867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 104/208] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:29 2022] GET /identity/v3/role_assignments?user.id=328674bcbff943fb878160494186b1b4&scope.project.id=81be097728134d76957fdb5755382cc2&role.id=4b22e77ca0b449f3b83e08e32d811efb => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:30.550762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555957b7-1bbc-4bc8-82a4-7b59f1234875 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:30.551650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555957b7-1bbc-4bc8-82a4-7b59f1234875 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:30.551650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555957b7-1bbc-4bc8-82a4-7b59f1234875 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:30.553038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 105/209] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:30.560686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8789aa9-ab17-4842-8373-a652712bca61 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:30.561742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8789aa9-ab17-4842-8373-a652712bca61 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:30.562235 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8789aa9-ab17-4842-8373-a652712bca61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:30.584630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f8789aa9-ab17-4842-8373-a652712bca61 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:30.586382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f8789aa9-ab17-4842-8373-a652712bca61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:30.599349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:30.675585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 105/210] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:59:30.689306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c7db5-a817-4347-893d-3c697a26429f None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:30.689671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c7db5-a817-4347-893d-3c697a26429f None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:30.689868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c7db5-a817-4347-893d-3c697a26429f None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:30.691304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 106/211] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:30.697850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10e82a-c007-4a97-ad0b-634f069395d8 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:30.698183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10e82a-c007-4a97-ad0b-634f069395d8 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:30.698427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10e82a-c007-4a97-ad0b-634f069395d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:30.719763 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-bd10e82a-c007-4a97-ad0b-634f069395d8 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:30.721360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bd10e82a-c007-4a97-ad0b-634f069395d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:30.806035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 106/212] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:59:30.975398 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c7db5-a817-4347-893d-3c697a26429f None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:30.994086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:31.100012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10e82a-c007-4a97-ad0b-634f069395d8 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:31.191070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:31.197969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c7db5-a817-4347-893d-3c697a26429f None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:31.221717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10e82a-c007-4a97-ad0b-634f069395d8 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:31.264984 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033bdf8-ab25-448a-8065-d9a18badcc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:31.265413 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3db16-2102-463a-b760-942cf095cf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:31.265654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033bdf8-ab25-448a-8065-d9a18badcc22 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:31.265938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033bdf8-ab25-448a-8065-d9a18badcc22 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:31.266212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033bdf8-ab25-448a-8065-d9a18badcc22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:31.267045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3db16-2102-463a-b760-942cf095cf45 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:31.267743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3db16-2102-463a-b760-942cf095cf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:31.268750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3db16-2102-463a-b760-942cf095cf45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:31.271380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3db16-2102-463a-b760-942cf095cf45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:31.274678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3db16-2102-463a-b760-942cf095cf45 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:31.278949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033bdf8-ab25-448a-8065-d9a18badcc22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:31.280267 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 107/213] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:59:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-5033bdf8-ab25-448a-8065-d9a18badcc22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:31.288777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:31.289756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:31.290551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:31.290551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:31.290551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:31.290551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:31.290551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:31.290551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:31.290551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:31.290551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:31.290551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:31.290551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:31.290551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:31.290551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:31.291237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3db16-2102-463a-b760-942cf095cf45 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:31.291311 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 107/214] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:31.291882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033bdf8-ab25-448a-8065-d9a18badcc22 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:31.380739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:31.430910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033bdf8-ab25-448a-8065-d9a18badcc22 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:31.449266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3db16-2102-463a-b760-942cf095cf45 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:31.501447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbfc13-304f-41cf-9f08-f8fd74d020a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:31.502422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbfc13-304f-41cf-9f08-f8fd74d020a3 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:31.502866 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbfc13-304f-41cf-9f08-f8fd74d020a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:31.503624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbfc13-304f-41cf-9f08-f8fd74d020a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:31.504652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bbfc13-304f-41cf-9f08-f8fd74d020a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:31.505134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bbfc13-304f-41cf-9f08-f8fd74d020a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:31.506848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bbfc13-304f-41cf-9f08-f8fd74d020a3 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:31.514228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf14b77-e082-4333-8577-0e69dd8b5835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:31.515383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf14b77-e082-4333-8577-0e69dd8b5835 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:31.515891 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf14b77-e082-4333-8577-0e69dd8b5835 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:31.516372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf14b77-e082-4333-8577-0e69dd8b5835 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:31.517656 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf14b77-e082-4333-8577-0e69dd8b5835 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:31.518830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 108/215] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:31.520888 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf14b77-e082-4333-8577-0e69dd8b5835 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:31.531176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbfc13-304f-41cf-9f08-f8fd74d020a3 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:31.552271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:31.564427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 108/216] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 12:59:31 2022] POST /identity/v3/users => generated 297 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:59:31.749927 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbfc13-304f-41cf-9f08-f8fd74d020a3 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:31.811594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:31.821727 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66e6be-1403-4050-938d-23c0a811478e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:31.822854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66e6be-1403-4050-938d-23c0a811478e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:31.823406 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66e6be-1403-4050-938d-23c0a811478e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:31.823915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66e6be-1403-4050-938d-23c0a811478e None admin] PATH_INFO: `/v3/users/10cbdc82574341fc8a244bc673d7d584` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:31.852513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66e6be-1403-4050-938d-23c0a811478e None admin] RBAC: Authorizing `identity:get_user(user_id=10cbdc82574341fc8a244bc673d7d584)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:31.855535 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66e6be-1403-4050-938d-23c0a811478e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:31.859735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 109/217] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:31 2022] GET /identity/v3/users/10cbdc82574341fc8a244bc673d7d584 => generated 297 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:31.869003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf14b77-e082-4333-8577-0e69dd8b5835 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:32.046192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf14b77-e082-4333-8577-0e69dd8b5835 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:32.071680 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22010047-a98e-4b2c-b9fc-bfe0d71bcc3a None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:32.072701 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22010047-a98e-4b2c-b9fc-bfe0d71bcc3a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:32.073444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22010047-a98e-4b2c-b9fc-bfe0d71bcc3a None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:32.077333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 110/218] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:32.087927 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70ce01-01d9-4464-839d-92352209f6dd None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:32.088786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70ce01-01d9-4464-839d-92352209f6dd None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:32.089536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70ce01-01d9-4464-839d-92352209f6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:32.107749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa1860-c599-4a38-b23a-b8bac61f9222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:32.107749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa1860-c599-4a38-b23a-b8bac61f9222 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:32.108395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa1860-c599-4a38-b23a-b8bac61f9222 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:32.108671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa1860-c599-4a38-b23a-b8bac61f9222 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:32.126562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-7a70ce01-01d9-4464-839d-92352209f6dd None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:32.128688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7a70ce01-01d9-4464-839d-92352209f6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:32.255577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:32.263809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 111/219] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:32.264399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa1860-c599-4a38-b23a-b8bac61f9222 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:32.266156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa1860-c599-4a38-b23a-b8bac61f9222 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:32.470212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 109/220] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:32.476045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70ce01-01d9-4464-839d-92352209f6dd None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:32.656830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70ce01-01d9-4464-839d-92352209f6dd None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:32.699642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:32.731047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c9bbc-5727-46f7-8620-34f3ccc3a876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:32.732860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c9bbc-5727-46f7-8620-34f3ccc3a876 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:32.733373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c9bbc-5727-46f7-8620-34f3ccc3a876 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:32.734008 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c9bbc-5727-46f7-8620-34f3ccc3a876 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:32.736234 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c9bbc-5727-46f7-8620-34f3ccc3a876 None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:32.739250 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c9bbc-5727-46f7-8620-34f3ccc3a876 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:32.743079 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 112/221] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:32 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:32.753699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa1860-c599-4a38-b23a-b8bac61f9222 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:32.797226 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c9bbc-5727-46f7-8620-34f3ccc3a876 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:33.025716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:33.035505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa1860-c599-4a38-b23a-b8bac61f9222 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:33.048458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c9bbc-5727-46f7-8620-34f3ccc3a876 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:33.090253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a4853-99ee-47f2-89f9-de3f70cfdb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:33.091584 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a4853-99ee-47f2-89f9-de3f70cfdb55 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:33.092090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a4853-99ee-47f2-89f9-de3f70cfdb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:33.092564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a4853-99ee-47f2-89f9-de3f70cfdb55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:33.105459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a4853-99ee-47f2-89f9-de3f70cfdb55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:33.112110 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49a720-c8b6-4562-be4d-d445c8f0a336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-279a4853-99ee-47f2-89f9-de3f70cfdb55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:33.112390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:33.114451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:33.116546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:33.116546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:33.116546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:33.116546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:33.116546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:33.116546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:33.116546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:33.116546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:33.116546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:33.116546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:33.116546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:59:33.116546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:33.116546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 113/222] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 12:59:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:33.119146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaae7b-5ba4-4309-874c-116d72ca0220 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:33.119146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaae7b-5ba4-4309-874c-116d72ca0220 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:33.119146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaae7b-5ba4-4309-874c-116d72ca0220 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:33.120063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49a720-c8b6-4562-be4d-d445c8f0a336 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:33.120894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49a720-c8b6-4562-be4d-d445c8f0a336 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:33.121673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49a720-c8b6-4562-be4d-d445c8f0a336 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:33.124050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49a720-c8b6-4562-be4d-d445c8f0a336 None admin] RBAC: Adding query filter params (role.id=86f185569d134d80841142ae8e9d3f47, scope.project.id=717d2ed44c684799a826e03b360f281b, user.id=10cbdc82574341fc8a244bc673d7d584) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:33.124918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49a720-c8b6-4562-be4d-d445c8f0a336 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:33.126201 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 114/223] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:33.127367 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaae7b-5ba4-4309-874c-116d72ca0220 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:33.129186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49a720-c8b6-4562-be4d-d445c8f0a336 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:33.142136 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 110/224] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:32 2022] GET /identity/v3/role_assignments?user.id=10cbdc82574341fc8a244bc673d7d584&scope.project.id=717d2ed44c684799a826e03b360f281b&role.id=86f185569d134d80841142ae8e9d3f47 => generated 259 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:33.150249 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f45ff7-41d1-436d-8da7-4c832e4391b8 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:33.151283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f45ff7-41d1-436d-8da7-4c832e4391b8 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:33.152151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f45ff7-41d1-436d-8da7-4c832e4391b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:33.197338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:33.216847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a3f45ff7-41d1-436d-8da7-4c832e4391b8 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:33.225946 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a3f45ff7-41d1-436d-8da7-4c832e4391b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:33.271810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaae7b-5ba4-4309-874c-116d72ca0220 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:33.321344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094afa84-95ee-4dc6-a4aa-1ed71451a7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:33.321971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094afa84-95ee-4dc6-a4aa-1ed71451a7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:33.322245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094afa84-95ee-4dc6-a4aa-1ed71451a7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:33.322508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094afa84-95ee-4dc6-a4aa-1ed71451a7f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:33.324217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094afa84-95ee-4dc6-a4aa-1ed71451a7f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:33.324519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094afa84-95ee-4dc6-a4aa-1ed71451a7f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:33.326583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094afa84-95ee-4dc6-a4aa-1ed71451a7f2 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:33.351627 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 115/225] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:33.358893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094afa84-95ee-4dc6-a4aa-1ed71451a7f2 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:33.421850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 111/226] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:33.549910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094afa84-95ee-4dc6-a4aa-1ed71451a7f2 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:33.572983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:33.633142 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e56eb-6059-4f0a-a05d-abd331be52e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:33.634237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e56eb-6059-4f0a-a05d-abd331be52e1 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:33.634718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e56eb-6059-4f0a-a05d-abd331be52e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:33.635204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e56eb-6059-4f0a-a05d-abd331be52e1 None admin] PATH_INFO: `/v3/users/1dd14697d94246a284b0106fcc1b4f20` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:33.637372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e56eb-6059-4f0a-a05d-abd331be52e1 None admin] RBAC: Authorizing `identity:get_user(user_id=1dd14697d94246a284b0106fcc1b4f20)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:33.640564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e56eb-6059-4f0a-a05d-abd331be52e1 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:33.645509 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 116/227] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:33 2022] GET /identity/v3/users/1dd14697d94246a284b0106fcc1b4f20 => generated 283 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:33.652884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f45ff7-41d1-436d-8da7-4c832e4391b8 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:33.766308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e56eb-6059-4f0a-a05d-abd331be52e1 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:33.915647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:33.942982 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f45ff7-41d1-436d-8da7-4c832e4391b8 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:34.074459 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b34de3-cfb7-41a1-8f25-ba95a016254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:34.079149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b34de3-cfb7-41a1-8f25-ba95a016254a None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:34.079149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b34de3-cfb7-41a1-8f25-ba95a016254a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:34.079149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b34de3-cfb7-41a1-8f25-ba95a016254a None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:34.082329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b34de3-cfb7-41a1-8f25-ba95a016254a None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:34.095144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b34de3-cfb7-41a1-8f25-ba95a016254a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:34.102242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 112/228] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:33 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:34.109304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b34de3-cfb7-41a1-8f25-ba95a016254a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:34.137561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e56eb-6059-4f0a-a05d-abd331be52e1 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:34.273231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa4816-6f05-458d-9bc2-7ce009bcce99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:34.274213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa4816-6f05-458d-9bc2-7ce009bcce99 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:34.274681 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa4816-6f05-458d-9bc2-7ce009bcce99 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:34.275200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa4816-6f05-458d-9bc2-7ce009bcce99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:34.304648 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa4816-6f05-458d-9bc2-7ce009bcce99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-c1aa4816-6f05-458d-9bc2-7ce009bcce99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:34.316971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:34.318722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:34.320100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:34.320100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:34.320100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:34.320100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:34.320100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:34.320100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:34.320100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:34.320100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:34.320100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:34.320100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:34.320100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:34.320100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:34.320100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 117/229] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 12:59:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 554 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:34.329303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa4816-6f05-458d-9bc2-7ce009bcce99 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:34.444381 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:34.497509 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b34de3-cfb7-41a1-8f25-ba95a016254a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:34.576906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa4816-6f05-458d-9bc2-7ce009bcce99 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:34.595252 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf4d6c-860d-43b4-9976-9cc0bfe010fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:34.596939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf4d6c-860d-43b4-9976-9cc0bfe010fb None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:34.602519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf4d6c-860d-43b4-9976-9cc0bfe010fb None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:34.603061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf4d6c-860d-43b4-9976-9cc0bfe010fb None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b/users/1dd14697d94246a284b0106fcc1b4f20/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:34.607534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf4d6c-860d-43b4-9976-9cc0bfe010fb None admin] RBAC: Authorizing `identity:create_grant(project_id=717d2ed44c684799a826e03b360f281b, user_id=1dd14697d94246a284b0106fcc1b4f20, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:34.610861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf4d6c-860d-43b4-9976-9cc0bfe010fb None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:34.654212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 113/230] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 12:59:34 2022] PUT /identity/v3/projects/717d2ed44c684799a826e03b360f281b/users/1dd14697d94246a284b0106fcc1b4f20/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 12:59:34.708442 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e16383-e632-4bb5-9747-31c224d70e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:34.709995 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e16383-e632-4bb5-9747-31c224d70e96 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:34.710734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e16383-e632-4bb5-9747-31c224d70e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:34.711498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e16383-e632-4bb5-9747-31c224d70e96 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:34.713071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e16383-e632-4bb5-9747-31c224d70e96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:34.713939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e16383-e632-4bb5-9747-31c224d70e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:34.717520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e16383-e632-4bb5-9747-31c224d70e96 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:34.739276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 118/231] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 12:59:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:34.745273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf4d6c-860d-43b4-9976-9cc0bfe010fb None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:34.821297 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:35.006021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf4d6c-860d-43b4-9976-9cc0bfe010fb None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:35.103920 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae7135-1a37-49a6-b54f-5f11c83f23cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:35.104597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae7135-1a37-49a6-b54f-5f11c83f23cd None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:35.104885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae7135-1a37-49a6-b54f-5f11c83f23cd None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:35.105166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae7135-1a37-49a6-b54f-5f11c83f23cd None admin] PATH_INFO: `/v3/users/1dd14697d94246a284b0106fcc1b4f20` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:35.106936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae7135-1a37-49a6-b54f-5f11c83f23cd None admin] RBAC: Authorizing `identity:get_user(user_id=1dd14697d94246a284b0106fcc1b4f20)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:35.108505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae7135-1a37-49a6-b54f-5f11c83f23cd None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:35.110204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 114/232] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:34 2022] GET /identity/v3/users/1dd14697d94246a284b0106fcc1b4f20 => generated 283 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:35.119916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e16383-e632-4bb5-9747-31c224d70e96 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:35.174855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7a180-6161-4817-a255-42de17717fc7 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:35.175331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7a180-6161-4817-a255-42de17717fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:35.175914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7a180-6161-4817-a255-42de17717fc7 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:35.177515 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 115/233] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:35.185873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289a8b4-9cb4-4bc1-9a31-90216ac6b4f5 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:35.185873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289a8b4-9cb4-4bc1-9a31-90216ac6b4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:35.185873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289a8b4-9cb4-4bc1-9a31-90216ac6b4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:35.210880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-e289a8b4-9cb4-4bc1-9a31-90216ac6b4f5 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:35.213785 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e289a8b4-9cb4-4bc1-9a31-90216ac6b4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:35.398002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 116/234] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:59:35.403001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9ed71-cda7-4898-b04e-11cdefd3cf3c None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:35.404120 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9ed71-cda7-4898-b04e-11cdefd3cf3c None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:35.404576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9ed71-cda7-4898-b04e-11cdefd3cf3c None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:35.406941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 117/235] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:59:35.417827 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:35.421671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a817c28-4aa7-4686-8d84-e4fc936f5372 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:35.422125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a817c28-4aa7-4686-8d84-e4fc936f5372 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:35.422612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a817c28-4aa7-4686-8d84-e4fc936f5372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:35.429292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e16383-e632-4bb5-9747-31c224d70e96 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:35.479723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-4a817c28-4aa7-4686-8d84-e4fc936f5372 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:35.482204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4a817c28-4aa7-4686-8d84-e4fc936f5372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:35.527680 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843f59d-79e8-4af3-8e8d-7d340a6013dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:35.529392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843f59d-79e8-4af3-8e8d-7d340a6013dc None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:35.529890 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843f59d-79e8-4af3-8e8d-7d340a6013dc None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:35.530386 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843f59d-79e8-4af3-8e8d-7d340a6013dc None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:35.532626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843f59d-79e8-4af3-8e8d-7d340a6013dc None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:35.535991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843f59d-79e8-4af3-8e8d-7d340a6013dc None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:35.539619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 119/236] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:35 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:35.542668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:35.551206 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843f59d-79e8-4af3-8e8d-7d340a6013dc None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:35.704868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 118/237] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:35.730400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a817c28-4aa7-4686-8d84-e4fc936f5372 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:35.773358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843f59d-79e8-4af3-8e8d-7d340a6013dc None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:35.815820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ced3f-99f5-4cfb-ac10-2b546fe8fdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:35.816522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ced3f-99f5-4cfb-ac10-2b546fe8fdf5 None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:35.816804 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ced3f-99f5-4cfb-ac10-2b546fe8fdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:35.817077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ced3f-99f5-4cfb-ac10-2b546fe8fdf5 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b/users/1dd14697d94246a284b0106fcc1b4f20/roles/649721c72e5d40deabd3512c529ffdf4` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:35.819739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ced3f-99f5-4cfb-ac10-2b546fe8fdf5 None admin] RBAC: Authorizing `identity:create_grant(project_id=717d2ed44c684799a826e03b360f281b, user_id=1dd14697d94246a284b0106fcc1b4f20, role_id=649721c72e5d40deabd3512c529ffdf4)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:35.821418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ced3f-99f5-4cfb-ac10-2b546fe8fdf5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:35.858449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 120/238] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 12:59:35 2022] PUT /identity/v3/projects/717d2ed44c684799a826e03b360f281b/users/1dd14697d94246a284b0106fcc1b4f20/roles/649721c72e5d40deabd3512c529ffdf4 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:59:35.979177 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a817c28-4aa7-4686-8d84-e4fc936f5372 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:36.049730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007697a-cf00-4a0f-8eb0-94a465948ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:36.052187 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007697a-cf00-4a0f-8eb0-94a465948ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:36.053036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007697a-cf00-4a0f-8eb0-94a465948ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:36.053805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007697a-cf00-4a0f-8eb0-94a465948ad5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:36.056379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007697a-cf00-4a0f-8eb0-94a465948ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:36.060276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007697a-cf00-4a0f-8eb0-94a465948ad5 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:36.064480 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 119/239] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:59:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:36.072251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ced3f-99f5-4cfb-ac10-2b546fe8fdf5 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:36.141764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:36.274879 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007697a-cf00-4a0f-8eb0-94a465948ad5 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:36.285993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ced3f-99f5-4cfb-ac10-2b546fe8fdf5 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:36.320849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:36.373650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55e910-4dd2-4df5-bd7f-f8952a8284c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:36.375249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e910-4dd2-4df5-bd7f-f8952a8284c9 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:36.376029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e910-4dd2-4df5-bd7f-f8952a8284c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:36.376762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e910-4dd2-4df5-bd7f-f8952a8284c9 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:36.378223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55e910-4dd2-4df5-bd7f-f8952a8284c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:36.382512 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55e910-4dd2-4df5-bd7f-f8952a8284c9 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:36.432386 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 121/240] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 12:59:36 2022] POST /identity/v3/users => generated 293 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:59:36.542388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007697a-cf00-4a0f-8eb0-94a465948ad5 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:36.619732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262a261-e341-44a8-855c-3fffd0a5010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:36.621434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262a261-e341-44a8-855c-3fffd0a5010b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:36.622341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262a261-e341-44a8-855c-3fffd0a5010b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:36.623047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262a261-e341-44a8-855c-3fffd0a5010b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:36.643842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262a261-e341-44a8-855c-3fffd0a5010b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-c262a261-e341-44a8-855c-3fffd0a5010b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:36.653100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:36.655539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:36.656964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:36.656964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:36.656964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:36.656964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:36.656964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:36.656964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:36.656964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:36.656964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:36.656964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:36.656964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:36.656964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:36.656964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:36.661325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 120/241] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:36.671537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55e910-4dd2-4df5-bd7f-f8952a8284c9 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:36.880028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:36.922721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55e910-4dd2-4df5-bd7f-f8952a8284c9 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:36.994392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad34236-a762-4b6d-93f3-3c0549dda561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:36.995072 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad34236-a762-4b6d-93f3-3c0549dda561 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:36.995483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad34236-a762-4b6d-93f3-3c0549dda561 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:36.995804 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad34236-a762-4b6d-93f3-3c0549dda561 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:36.996636 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad34236-a762-4b6d-93f3-3c0549dda561 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:36.997558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad34236-a762-4b6d-93f3-3c0549dda561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:36.999026 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad34236-a762-4b6d-93f3-3c0549dda561 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:37.012711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 122/242] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:37.022370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262a261-e341-44a8-855c-3fffd0a5010b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:37.062874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000513c-9c17-4c15-b87a-16051c0fc4b6 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:37.063309 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000513c-9c17-4c15-b87a-16051c0fc4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:37.063446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000513c-9c17-4c15-b87a-16051c0fc4b6 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:37.064948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 123/243] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:59:37.071943 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d52bd-f018-4757-8498-a0963e285739 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:37.072373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d52bd-f018-4757-8498-a0963e285739 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:37.072751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d52bd-f018-4757-8498-a0963e285739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:37.095519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-fd2d52bd-f018-4757-8498-a0963e285739 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:37.096938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-fd2d52bd-f018-4757-8498-a0963e285739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:37.199224 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:37.213214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 124/244] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:59:37.248161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262a261-e341-44a8-855c-3fffd0a5010b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:37.314996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d96c9-2a89-4b61-b82d-1c987c6553d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:37.316076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d96c9-2a89-4b61-b82d-1c987c6553d4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:37.316574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d96c9-2a89-4b61-b82d-1c987c6553d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:37.317035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d96c9-2a89-4b61-b82d-1c987c6553d4 None admin] PATH_INFO: `/v3/users/1a93a8336a85455e97e9455660d180c1` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:37.349011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d96c9-2a89-4b61-b82d-1c987c6553d4 None admin] RBAC: Authorizing `identity:get_user(user_id=1a93a8336a85455e97e9455660d180c1)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:37.351954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d96c9-2a89-4b61-b82d-1c987c6553d4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:37.355206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 121/245] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:37 2022] GET /identity/v3/users/1a93a8336a85455e97e9455660d180c1 => generated 297 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:37.364666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d52bd-f018-4757-8498-a0963e285739 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:37.475510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:37.506116 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d52bd-f018-4757-8498-a0963e285739 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:37.567516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a85fb-2e32-472f-aa08-c7283e34bfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:37.568162 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a85fb-2e32-472f-aa08-c7283e34bfc0 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:37.568436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a85fb-2e32-472f-aa08-c7283e34bfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:37.568711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a85fb-2e32-472f-aa08-c7283e34bfc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:37.694033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a85fb-2e32-472f-aa08-c7283e34bfc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:37.695146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a85fb-2e32-472f-aa08-c7283e34bfc0 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:37.776941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d96c9-2a89-4b61-b82d-1c987c6553d4 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:37.827713 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:37.859248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 125/246] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:37.867484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a85fb-2e32-472f-aa08-c7283e34bfc0 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:38.101869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a85fb-2e32-472f-aa08-c7283e34bfc0 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:38.189994 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d96c9-2a89-4b61-b82d-1c987c6553d4 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:38.204875 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff0040-e449-4553-b382-6f6ca0abbdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:38.205540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff0040-e449-4553-b382-6f6ca0abbdbd None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:38.205818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff0040-e449-4553-b382-6f6ca0abbdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:38.206100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff0040-e449-4553-b382-6f6ca0abbdbd None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:38.209530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff0040-e449-4553-b382-6f6ca0abbdbd None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:38.211029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff0040-e449-4553-b382-6f6ca0abbdbd None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:38.213096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 126/247] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:37 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:38.222405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff0040-e449-4553-b382-6f6ca0abbdbd None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:38.308270 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a2ac1-4900-44b2-96f0-f4b42a3b4757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:38.310550 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a2ac1-4900-44b2-96f0-f4b42a3b4757 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:38.311370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a2ac1-4900-44b2-96f0-f4b42a3b4757 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:38.312182 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a2ac1-4900-44b2-96f0-f4b42a3b4757 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:38.331138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:38.357771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a2ac1-4900-44b2-96f0-f4b42a3b4757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:38.367654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-4e1a2ac1-4900-44b2-96f0-f4b42a3b4757 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:38.376343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:38.378153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:38.380900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:38.380900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:38.380900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:38.380900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:38.380900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:38.380900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:38.380900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:38.380900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:38.380900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:38.380900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:38.380900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:38.380900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:38.380900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 122/248] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 603 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:38.390877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a2ac1-4900-44b2-96f0-f4b42a3b4757 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:38.540060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff0040-e449-4553-b382-6f6ca0abbdbd None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:38.626551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e630c10-a4a1-482f-8057-e32d5b7ac9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:38.627578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e630c10-a4a1-482f-8057-e32d5b7ac9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:38.628039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e630c10-a4a1-482f-8057-e32d5b7ac9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:38.629568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e630c10-a4a1-482f-8057-e32d5b7ac9d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:38.631048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e630c10-a4a1-482f-8057-e32d5b7ac9d2 None admin] RBAC: Adding query filter params (role.id=4b22e77ca0b449f3b83e08e32d811efb, scope.project.id=717d2ed44c684799a826e03b360f281b, user.id=1a93a8336a85455e97e9455660d180c1) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:38.631611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e630c10-a4a1-482f-8057-e32d5b7ac9d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:38.634810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e630c10-a4a1-482f-8057-e32d5b7ac9d2 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:38.654065 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 127/249] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:38 2022] GET /identity/v3/role_assignments?user.id=1a93a8336a85455e97e9455660d180c1&scope.project.id=717d2ed44c684799a826e03b360f281b&role.id=4b22e77ca0b449f3b83e08e32d811efb => generated 259 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:38.788612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a2ac1-4900-44b2-96f0-f4b42a3b4757 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:38.865256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a706be-5d7e-48b5-b8c3-23d69bbb6616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:38.866327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a706be-5d7e-48b5-b8c3-23d69bbb6616 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:38.866835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a706be-5d7e-48b5-b8c3-23d69bbb6616 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:38.867366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a706be-5d7e-48b5-b8c3-23d69bbb6616 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:38.868664 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a706be-5d7e-48b5-b8c3-23d69bbb6616 None admin] RBAC: Adding query filter params (name=member) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:38.869208 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a706be-5d7e-48b5-b8c3-23d69bbb6616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:38.872765 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a706be-5d7e-48b5-b8c3-23d69bbb6616 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:38.876084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b1da-6eb5-4a39-aa18-b2be75620eb7 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:38.876749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b1da-6eb5-4a39-aa18-b2be75620eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:38.877126 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b1da-6eb5-4a39-aa18-b2be75620eb7 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:38.884433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 128/250] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:38 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:38.889647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 123/251] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:38.895210 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f161-e944-4568-bda5-5da10f130f7e None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:38.895698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f161-e944-4568-bda5-5da10f130f7e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:38.896180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f161-e944-4568-bda5-5da10f130f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:38.898644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a706be-5d7e-48b5-b8c3-23d69bbb6616 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:38.955456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-6628f161-e944-4568-bda5-5da10f130f7e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:38.968423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6628f161-e944-4568-bda5-5da10f130f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:39.110290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:39.238617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a706be-5d7e-48b5-b8c3-23d69bbb6616 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:39.249920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 129/252] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:39.305452 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7f752-6ca2-4594-973d-db41a9fd1cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:39.306574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7f752-6ca2-4594-973d-db41a9fd1cda None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:39.307087 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7f752-6ca2-4594-973d-db41a9fd1cda None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:39.307613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7f752-6ca2-4594-973d-db41a9fd1cda None admin] PATH_INFO: `/v3/users/10cbdc82574341fc8a244bc673d7d584` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:39.310237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7f752-6ca2-4594-973d-db41a9fd1cda None admin] RBAC: Authorizing `identity:get_user(user_id=10cbdc82574341fc8a244bc673d7d584)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:39.312806 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7f752-6ca2-4594-973d-db41a9fd1cda None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:39.315580 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 124/253] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:38 2022] GET /identity/v3/users/10cbdc82574341fc8a244bc673d7d584 => generated 297 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:39.324032 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f161-e944-4568-bda5-5da10f130f7e None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:39.431825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:39.512012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f161-e944-4568-bda5-5da10f130f7e None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:39.563005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f70a1e-38d7-4872-9363-01a8f2962778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:39.563978 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f70a1e-38d7-4872-9363-01a8f2962778 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:39.563978 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f70a1e-38d7-4872-9363-01a8f2962778 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:39.564231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f70a1e-38d7-4872-9363-01a8f2962778 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:39.565487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f70a1e-38d7-4872-9363-01a8f2962778 None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:39.567131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f70a1e-38d7-4872-9363-01a8f2962778 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:39.569282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 130/254] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:39 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:39.578755 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7f752-6ca2-4594-973d-db41a9fd1cda None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:39.906531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7f752-6ca2-4594-973d-db41a9fd1cda None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:39.952145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:39.969588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829f50a-7881-4e3d-9f5d-c9a424c44a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:39.970283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829f50a-7881-4e3d-9f5d-c9a424c44a11 None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:39.970580 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829f50a-7881-4e3d-9f5d-c9a424c44a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:39.970835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829f50a-7881-4e3d-9f5d-c9a424c44a11 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b/users/10cbdc82574341fc8a244bc673d7d584/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:39.976087 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829f50a-7881-4e3d-9f5d-c9a424c44a11 None admin] RBAC: Authorizing `identity:create_grant(project_id=717d2ed44c684799a826e03b360f281b, user_id=10cbdc82574341fc8a244bc673d7d584, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:39.977597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829f50a-7881-4e3d-9f5d-c9a424c44a11 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:39.996936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 125/255] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 12:59:39 2022] PUT /identity/v3/projects/717d2ed44c684799a826e03b360f281b/users/10cbdc82574341fc8a244bc673d7d584/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:59:40.069634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f70a1e-38d7-4872-9363-01a8f2962778 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:40.236004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f70a1e-38d7-4872-9363-01a8f2962778 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:40.245878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:40.289746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df6872-1040-47a3-b925-608ca2de3e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:40.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df6872-1040-47a3-b925-608ca2de3e59 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:40.290685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df6872-1040-47a3-b925-608ca2de3e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:40.290950 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df6872-1040-47a3-b925-608ca2de3e59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:40.311711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df6872-1040-47a3-b925-608ca2de3e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-c6df6872-1040-47a3-b925-608ca2de3e59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:40.316813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:40.317773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:40.318560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:40.318560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:40.318560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:40.318560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:40.318560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:40.318560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:40.318560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:40.318560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:40.318560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:40.318560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:40.318560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 12:59:40.318560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:40.318560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 131/256] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 12:59:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:40.325850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829f50a-7881-4e3d-9f5d-c9a424c44a11 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:40.564068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829f50a-7881-4e3d-9f5d-c9a424c44a11 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:40.654201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf2f34-2c15-4ad8-ac54-545c314bdc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:40.655364 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf2f34-2c15-4ad8-ac54-545c314bdc0e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:40.655858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf2f34-2c15-4ad8-ac54-545c314bdc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:40.656347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf2f34-2c15-4ad8-ac54-545c314bdc0e None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:40.657654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf2f34-2c15-4ad8-ac54-545c314bdc0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:40.658199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf2f34-2c15-4ad8-ac54-545c314bdc0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:40.660852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf2f34-2c15-4ad8-ac54-545c314bdc0e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:40.680134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 126/257] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:40.689424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df6872-1040-47a3-b925-608ca2de3e59 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:40.804777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75456e58-9f7d-45a6-8b08-81cde5ad75aa None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:40.805255 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75456e58-9f7d-45a6-8b08-81cde5ad75aa None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:40.805662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75456e58-9f7d-45a6-8b08-81cde5ad75aa None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:40.807705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 127/258] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:40.817572 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5966f6-8302-4869-ae01-b0e54bcdf9d4 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:40.817965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5966f6-8302-4869-ae01-b0e54bcdf9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:40.818291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5966f6-8302-4869-ae01-b0e54bcdf9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:40.877064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-ef5966f6-8302-4869-ae01-b0e54bcdf9d4 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:40.879524 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ef5966f6-8302-4869-ae01-b0e54bcdf9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:40.911937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df6872-1040-47a3-b925-608ca2de3e59 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:41.008626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ef950-6078-41b0-bfe1-347790892a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:41.010188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ef950-6078-41b0-bfe1-347790892a52 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:41.010915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ef950-6078-41b0-bfe1-347790892a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:41.011753 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ef950-6078-41b0-bfe1-347790892a52 None admin] PATH_INFO: `/v3/users/1dd14697d94246a284b0106fcc1b4f20` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:41.016473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ef950-6078-41b0-bfe1-347790892a52 None admin] RBAC: Authorizing `identity:get_user(user_id=1dd14697d94246a284b0106fcc1b4f20)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:41.021099 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:41.022696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ef950-6078-41b0-bfe1-347790892a52 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:41.025899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 132/259] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:40 2022] GET /identity/v3/users/1dd14697d94246a284b0106fcc1b4f20 => generated 283 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:41.048474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ef950-6078-41b0-bfe1-347790892a52 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:41.125683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 128/260] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:59:41.139221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:41.270441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ef950-6078-41b0-bfe1-347790892a52 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:41.335582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7adaf-4293-4ada-8b80-2133c6261f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:41.336534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7adaf-4293-4ada-8b80-2133c6261f6c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:41.337001 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7adaf-4293-4ada-8b80-2133c6261f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:41.337473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7adaf-4293-4ada-8b80-2133c6261f6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:41.474134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7adaf-4293-4ada-8b80-2133c6261f6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:41.476933 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7adaf-4293-4ada-8b80-2133c6261f6c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:41.566578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:41.733625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5966f6-8302-4869-ae01-b0e54bcdf9d4 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:41.743556 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 133/261] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 12:59:41.756612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7adaf-4293-4ada-8b80-2133c6261f6c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:42.049702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7adaf-4293-4ada-8b80-2133c6261f6c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:42.102421 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5966f6-8302-4869-ae01-b0e54bcdf9d4 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:42.103964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201ed23-9fa9-4d40-b1bd-5543e88468b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:42.105044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201ed23-9fa9-4d40-b1bd-5543e88468b8 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:42.105538 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201ed23-9fa9-4d40-b1bd-5543e88468b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:42.106029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201ed23-9fa9-4d40-b1bd-5543e88468b8 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:42.108795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201ed23-9fa9-4d40-b1bd-5543e88468b8 None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:42.111665 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201ed23-9fa9-4d40-b1bd-5543e88468b8 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:42.114949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 134/262] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:41 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:42.125654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458921e-aacf-4b74-9a46-cffdd1138dae None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:42.126157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458921e-aacf-4b74-9a46-cffdd1138dae None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:42.126632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458921e-aacf-4b74-9a46-cffdd1138dae None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:42.129570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 135/263] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:42 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:42.131810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0116885-6585-4e31-a1ca-07ffb091d1d8 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:42.132308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0116885-6585-4e31-a1ca-07ffb091d1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:42.132784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0116885-6585-4e31-a1ca-07ffb091d1d8 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:42.135395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 136/264] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:42.136844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0116885-6585-4e31-a1ca-07ffb091d1d8 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:42.149122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:42.154003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902186c8-8b6a-4603-8a7b-5d4555043f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:42.155065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902186c8-8b6a-4603-8a7b-5d4555043f87 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:42.155528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902186c8-8b6a-4603-8a7b-5d4555043f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:42.156020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902186c8-8b6a-4603-8a7b-5d4555043f87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:42.176118 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902186c8-8b6a-4603-8a7b-5d4555043f87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-902186c8-8b6a-4603-8a7b-5d4555043f87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:42.187397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:42.188386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 129/265] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 12:59:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 456 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:42.194200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a419-4a3b-4d1b-8a2a-502b44b65a29 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:42.194935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a419-4a3b-4d1b-8a2a-502b44b65a29 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:42.194935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a419-4a3b-4d1b-8a2a-502b44b65a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:42.214156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-be54a419-4a3b-4d1b-8a2a-502b44b65a29 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:42.216985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-be54a419-4a3b-4d1b-8a2a-502b44b65a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:42.282006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0116885-6585-4e31-a1ca-07ffb091d1d8 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:42.315808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:42.334820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f2f7b-89fe-448a-9a8b-398cbfaf89d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:42.336446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f2f7b-89fe-448a-9a8b-398cbfaf89d3 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:42.336914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f2f7b-89fe-448a-9a8b-398cbfaf89d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:42.338215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f2f7b-89fe-448a-9a8b-398cbfaf89d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:42.340185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f2f7b-89fe-448a-9a8b-398cbfaf89d3 None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=717d2ed44c684799a826e03b360f281b, user.id=1dd14697d94246a284b0106fcc1b4f20) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:42.341276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f2f7b-89fe-448a-9a8b-398cbfaf89d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:42.346020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f2f7b-89fe-448a-9a8b-398cbfaf89d3 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:42.349066 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 130/266] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:42.359615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 137/267] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:42 2022] GET /identity/v3/role_assignments?user.id=1dd14697d94246a284b0106fcc1b4f20&scope.project.id=717d2ed44c684799a826e03b360f281b&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:42.360358 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e9aaf-7b74-4f91-b9e5-4f1838203c7f None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:42.360898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e9aaf-7b74-4f91-b9e5-4f1838203c7f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:42.361095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f2f7b-89fe-448a-9a8b-398cbfaf89d3 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:42.361597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e9aaf-7b74-4f91-b9e5-4f1838203c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:42.406682 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-2a2e9aaf-7b74-4f91-b9e5-4f1838203c7f None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:42.408462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2a2e9aaf-7b74-4f91-b9e5-4f1838203c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:42.567679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:42.645293 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 131/268] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:42.647743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688ba17-045b-43ba-8936-f3ea00ea329f None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:42.648250 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688ba17-045b-43ba-8936-f3ea00ea329f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:42.650952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688ba17-045b-43ba-8936-f3ea00ea329f None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:42.661440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 132/269] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:42 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:42.671265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d243fd1-3325-4599-8149-c12a598f8ec0 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:42.671265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d243fd1-3325-4599-8149-c12a598f8ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:42.671265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d243fd1-3325-4599-8149-c12a598f8ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:42.695748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f2f7b-89fe-448a-9a8b-398cbfaf89d3 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:42.725167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-2d243fd1-3325-4599-8149-c12a598f8ec0 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:42.727360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2d243fd1-3325-4599-8149-c12a598f8ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:42.837045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41502215-d140-4802-8754-b27d33c92e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:42.837983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41502215-d140-4802-8754-b27d33c92e84 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:42.838412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41502215-d140-4802-8754-b27d33c92e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:42.838920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41502215-d140-4802-8754-b27d33c92e84 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:42.840225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41502215-d140-4802-8754-b27d33c92e84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:42.848697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41502215-d140-4802-8754-b27d33c92e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:42.851952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41502215-d140-4802-8754-b27d33c92e84 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:42.896563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 138/270] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 12:59:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 536 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:42.907486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41502215-d140-4802-8754-b27d33c92e84 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:42.987248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 133/271] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 07 12:59:43.026205 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d243fd1-3325-4599-8149-c12a598f8ec0 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:43.099938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:43.170139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41502215-d140-4802-8754-b27d33c92e84 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:43.234783 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:43.255643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7a2f5-e3f3-411b-b18e-b1fabf4c9a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:43.256304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7a2f5-e3f3-411b-b18e-b1fabf4c9a5d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:43.256581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7a2f5-e3f3-411b-b18e-b1fabf4c9a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:43.256852 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7a2f5-e3f3-411b-b18e-b1fabf4c9a5d None admin] PATH_INFO: `/v3/users/1dd14697d94246a284b0106fcc1b4f20` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:43.259142 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7a2f5-e3f3-411b-b18e-b1fabf4c9a5d None admin] RBAC: Authorizing `identity:get_user(user_id=1dd14697d94246a284b0106fcc1b4f20)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:43.260641 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7a2f5-e3f3-411b-b18e-b1fabf4c9a5d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:43.264395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 139/272] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:42 2022] GET /identity/v3/users/1dd14697d94246a284b0106fcc1b4f20 => generated 283 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:43.274241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7a2f5-e3f3-411b-b18e-b1fabf4c9a5d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:43.299156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d243fd1-3325-4599-8149-c12a598f8ec0 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:43.367257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0d7d3-5802-4f92-92b2-ea43673b0431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:43.367257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0d7d3-5802-4f92-92b2-ea43673b0431 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:43.367642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0d7d3-5802-4f92-92b2-ea43673b0431 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:43.368108 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0d7d3-5802-4f92-92b2-ea43673b0431 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:43.389901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0d7d3-5802-4f92-92b2-ea43673b0431 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-73c0d7d3-5802-4f92-92b2-ea43673b0431 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:43.400149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:43.401938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:43.404708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:43.404708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:43.404708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:43.404708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:43.404708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:43.404708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:43.404708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:43.404708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:43.404708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:43.404708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:43.404708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:43.404708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:43.404708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 134/273] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:59:43.405885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0d7d3-5802-4f92-92b2-ea43673b0431 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:43.449660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7a2f5-e3f3-411b-b18e-b1fabf4c9a5d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:43.514749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e40c5-8c0f-4593-a592-173469f6aa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:43.515833 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e40c5-8c0f-4593-a592-173469f6aa5d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:43.516450 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e40c5-8c0f-4593-a592-173469f6aa5d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:43.517010 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e40c5-8c0f-4593-a592-173469f6aa5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:43.524815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:43.544724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0d7d3-5802-4f92-92b2-ea43673b0431 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:43.582392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c39c00-f285-476d-895d-46c2f1697965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:43.583046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c39c00-f285-476d-895d-46c2f1697965 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:43.583348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c39c00-f285-476d-895d-46c2f1697965 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:43.583632 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c39c00-f285-476d-895d-46c2f1697965 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:43.584797 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c39c00-f285-476d-895d-46c2f1697965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:43.586268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c39c00-f285-476d-895d-46c2f1697965 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:43.593143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 135/274] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:59:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:43.593143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c39c00-f285-476d-895d-46c2f1697965 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:43.642389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:43.658090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e40c5-8c0f-4593-a592-173469f6aa5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:43.661122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e40c5-8c0f-4593-a592-173469f6aa5d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:43.735655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c39c00-f285-476d-895d-46c2f1697965 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:43.798973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27982b-261e-4dde-8420-699dca9a1c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:43.800090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27982b-261e-4dde-8420-699dca9a1c6e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:43.800615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27982b-261e-4dde-8420-699dca9a1c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:43.801126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27982b-261e-4dde-8420-699dca9a1c6e None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:43.802426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27982b-261e-4dde-8420-699dca9a1c6e None admin] RBAC: Adding query filter params (name=member) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:43.803006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27982b-261e-4dde-8420-699dca9a1c6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:43.805827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27982b-261e-4dde-8420-699dca9a1c6e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:43.820543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 140/275] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:43.821337 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e40c5-8c0f-4593-a592-173469f6aa5d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:43.829257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 136/276] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:43.829257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27982b-261e-4dde-8420-699dca9a1c6e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:43.850689 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:43.946039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27982b-261e-4dde-8420-699dca9a1c6e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:43.982039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:43.990047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d09d2-3634-46b5-8b97-a5516c3860dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:43.991017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d09d2-3634-46b5-8b97-a5516c3860dd None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:43.991465 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e40c5-8c0f-4593-a592-173469f6aa5d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:43.991556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d09d2-3634-46b5-8b97-a5516c3860dd None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:43.991931 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d09d2-3634-46b5-8b97-a5516c3860dd None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:43.993038 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d09d2-3634-46b5-8b97-a5516c3860dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:43.995582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d09d2-3634-46b5-8b97-a5516c3860dd None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:44.027621 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 137/277] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 12:59:43 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 12:59:44.029432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d09d2-3634-46b5-8b97-a5516c3860dd None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:44.045292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2550a7-58ce-4106-949a-8c85e25590f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:44.046134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2550a7-58ce-4106-949a-8c85e25590f3 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:44.046413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2550a7-58ce-4106-949a-8c85e25590f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:44.046735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2550a7-58ce-4106-949a-8c85e25590f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:44.061381 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2550a7-58ce-4106-949a-8c85e25590f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-8e2550a7-58ce-4106-949a-8c85e25590f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:44.066570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:44.068456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:44.070213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:44.070213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:44.070213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:44.070213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:44.070213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:44.070213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:44.070213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:44.070213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:44.070213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:44.070213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:44.070213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:44.070213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:44.071829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 141/278] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:44.072030 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2550a7-58ce-4106-949a-8c85e25590f3 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:44.170514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:44.213589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d09d2-3634-46b5-8b97-a5516c3860dd None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:44.261354 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695da072-1793-4023-9246-e95796c8fe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:44.262015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695da072-1793-4023-9246-e95796c8fe55 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:44.262812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695da072-1793-4023-9246-e95796c8fe55 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:44.263091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695da072-1793-4023-9246-e95796c8fe55 None admin] PATH_INFO: `/v3/users/16ae927e7ced46c79a8a48f9d7dc0ab9` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:44.270433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2550a7-58ce-4106-949a-8c85e25590f3 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:44.278841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695da072-1793-4023-9246-e95796c8fe55 None admin] RBAC: Authorizing `identity:get_user(user_id=16ae927e7ced46c79a8a48f9d7dc0ab9)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:44.280431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695da072-1793-4023-9246-e95796c8fe55 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:44.282188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 138/279] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:44 2022] GET /identity/v3/users/16ae927e7ced46c79a8a48f9d7dc0ab9 => generated 293 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:44.283479 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238db2e3-9058-4a02-a0b6-21529febd92a None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:44.283787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238db2e3-9058-4a02-a0b6-21529febd92a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:44.284143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238db2e3-9058-4a02-a0b6-21529febd92a None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:44.285622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 139/280] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:44.286341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238db2e3-9058-4a02-a0b6-21529febd92a None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:44.319824 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039460c-51d1-4a8d-9855-476775f6f8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:44.320520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039460c-51d1-4a8d-9855-476775f6f8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:44.320827 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039460c-51d1-4a8d-9855-476775f6f8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:44.321100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039460c-51d1-4a8d-9855-476775f6f8f5 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:44.323153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039460c-51d1-4a8d-9855-476775f6f8f5 None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:44.326278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039460c-51d1-4a8d-9855-476775f6f8f5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:44.329707 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 142/281] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:44 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:44.330918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039460c-51d1-4a8d-9855-476775f6f8f5 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:44.378750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:44.455427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238db2e3-9058-4a02-a0b6-21529febd92a None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:44.502154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039460c-51d1-4a8d-9855-476775f6f8f5 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:44.503879 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:44.524035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922521a-a6d9-436d-94db-7da61fb5b841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:44.525289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922521a-a6d9-436d-94db-7da61fb5b841 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:44.525784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922521a-a6d9-436d-94db-7da61fb5b841 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:44.526283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922521a-a6d9-436d-94db-7da61fb5b841 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:44.528674 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922521a-a6d9-436d-94db-7da61fb5b841 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:44.528674 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922521a-a6d9-436d-94db-7da61fb5b841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:44.531285 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922521a-a6d9-436d-94db-7da61fb5b841 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:44.547070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 140/282] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:44.548768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b23edb-aad5-4ccd-80ce-86acdb20028e None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:44.549111 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b23edb-aad5-4ccd-80ce-86acdb20028e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:44.549409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b23edb-aad5-4ccd-80ce-86acdb20028e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:44.563253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eddcb3-a86e-4a86-ab16-e8fada34f1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:44.564204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eddcb3-a86e-4a86-ab16-e8fada34f1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:44.564625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eddcb3-a86e-4a86-ab16-e8fada34f1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:44.565684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eddcb3-a86e-4a86-ab16-e8fada34f1a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:44.566988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eddcb3-a86e-4a86-ab16-e8fada34f1a5 None admin] RBAC: Adding query filter params (role.id=86f185569d134d80841142ae8e9d3f47, user.id=16ae927e7ced46c79a8a48f9d7dc0ab9, scope.system=all) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:44.567492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eddcb3-a86e-4a86-ab16-e8fada34f1a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:44.569212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eddcb3-a86e-4a86-ab16-e8fada34f1a5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:44.570856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-44b23edb-aad5-4ccd-80ce-86acdb20028e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:44.572135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-44b23edb-aad5-4ccd-80ce-86acdb20028e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:44.577305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 143/283] 10.222.0.61 () {56 vars in 1308 bytes} [Wed Sep 7 12:59:44 2022] GET /identity/v3/role_assignments?user.id=16ae927e7ced46c79a8a48f9d7dc0ab9&scope.system=all&role.id=86f185569d134d80841142ae8e9d3f47 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:44.585524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eddcb3-a86e-4a86-ab16-e8fada34f1a5 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:44.746376 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 141/284] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:44.747312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b23edb-aad5-4ccd-80ce-86acdb20028e None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:44.796876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:44.832111 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eddcb3-a86e-4a86-ab16-e8fada34f1a5 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:44.877015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e1130-291f-4337-8e6e-f93133f5860d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:44.878045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1130-291f-4337-8e6e-f93133f5860d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:44.878800 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1130-291f-4337-8e6e-f93133f5860d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:44.879345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1130-291f-4337-8e6e-f93133f5860d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:44.880442 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e1130-291f-4337-8e6e-f93133f5860d None admin] RBAC: Adding query filter params (role.id=649721c72e5d40deabd3512c529ffdf4, scope.project.id=717d2ed44c684799a826e03b360f281b, user.id=1dd14697d94246a284b0106fcc1b4f20) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:44.880968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e1130-291f-4337-8e6e-f93133f5860d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:44.882661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e1130-291f-4337-8e6e-f93133f5860d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:44.889906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 144/285] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:44 2022] GET /identity/v3/role_assignments?user.id=1dd14697d94246a284b0106fcc1b4f20&scope.project.id=717d2ed44c684799a826e03b360f281b&role.id=649721c72e5d40deabd3512c529ffdf4 => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:44.930948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b23edb-aad5-4ccd-80ce-86acdb20028e None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:44.961448 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:44.982398 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ce671-e8dc-478a-96ce-8c59c3362cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:44.983499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ce671-e8dc-478a-96ce-8c59c3362cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:44.984009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ce671-e8dc-478a-96ce-8c59c3362cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:44.984501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ce671-e8dc-478a-96ce-8c59c3362cf2 None admin] PATH_INFO: `/v3/users/1a93a8336a85455e97e9455660d180c1` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:44.986002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ce671-e8dc-478a-96ce-8c59c3362cf2 None admin] RBAC: Authorizing `identity:get_user(user_id=1a93a8336a85455e97e9455660d180c1)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:44.987807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ce671-e8dc-478a-96ce-8c59c3362cf2 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:44.989881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 142/286] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:44 2022] GET /identity/v3/users/1a93a8336a85455e97e9455660d180c1 => generated 297 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:44.999239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e1130-291f-4337-8e6e-f93133f5860d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:45.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e1130-291f-4337-8e6e-f93133f5860d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:45.292910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9e6e3-1ab0-4260-bd53-33920b697a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:45.295809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9e6e3-1ab0-4260-bd53-33920b697a09 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:45.295809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9e6e3-1ab0-4260-bd53-33920b697a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:45.295809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9e6e3-1ab0-4260-bd53-33920b697a09 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:45.297167 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9e6e3-1ab0-4260-bd53-33920b697a09 None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:45.299963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9e6e3-1ab0-4260-bd53-33920b697a09 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:45.307754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 145/287] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:44 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:45.313636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ce671-e8dc-478a-96ce-8c59c3362cf2 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:45.453832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9e6e3-1ab0-4260-bd53-33920b697a09 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:45.527522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:45.587945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ce671-e8dc-478a-96ce-8c59c3362cf2 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:45.719024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27dd9b-501f-4a75-a927-7b6269c590cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:45.723773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27dd9b-501f-4a75-a927-7b6269c590cb None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:45.724217 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27dd9b-501f-4a75-a927-7b6269c590cb None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:45.724695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27dd9b-501f-4a75-a927-7b6269c590cb None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b/users/1a93a8336a85455e97e9455660d180c1/roles/4b22e77ca0b449f3b83e08e32d811efb` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:45.728353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27dd9b-501f-4a75-a927-7b6269c590cb None admin] RBAC: Authorizing `identity:create_grant(project_id=717d2ed44c684799a826e03b360f281b, user_id=1a93a8336a85455e97e9455660d180c1, role_id=4b22e77ca0b449f3b83e08e32d811efb)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:45.731551 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27dd9b-501f-4a75-a927-7b6269c590cb None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:45.755054 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9e6e3-1ab0-4260-bd53-33920b697a09 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:45.761807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 143/288] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 12:59:45 2022] PUT /identity/v3/projects/717d2ed44c684799a826e03b360f281b/users/1a93a8336a85455e97e9455660d180c1/roles/4b22e77ca0b449f3b83e08e32d811efb => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 12:59:45.853227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdec47-9403-4460-b2de-bd4186085b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:45.854272 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdec47-9403-4460-b2de-bd4186085b32 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:45.854730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdec47-9403-4460-b2de-bd4186085b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:45.855224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdec47-9403-4460-b2de-bd4186085b32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:45.886588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdec47-9403-4460-b2de-bd4186085b32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-28cdec47-9403-4460-b2de-bd4186085b32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:45.902558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:45.904148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:45.905040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:45.905040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:45.905040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:45.905040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:45.905040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:45.905040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:45.905040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:45.905040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:45.905040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:45.905040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:45.905040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:45.905040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:45.907027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 146/289] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 453 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:45.917409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27dd9b-501f-4a75-a927-7b6269c590cb None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:46.077800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:46.197772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27dd9b-501f-4a75-a927-7b6269c590cb None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:46.264425 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe433ae-2363-4e34-a080-fcf143289296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:46.266016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe433ae-2363-4e34-a080-fcf143289296 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:46.266786 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe433ae-2363-4e34-a080-fcf143289296 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:46.267611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe433ae-2363-4e34-a080-fcf143289296 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:46.269176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe433ae-2363-4e34-a080-fcf143289296 None admin] RBAC: Adding query filter params (name=member) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:46.269998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe433ae-2363-4e34-a080-fcf143289296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:46.272908 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe433ae-2363-4e34-a080-fcf143289296 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:46.290775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 144/290] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:46.301066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdec47-9403-4460-b2de-bd4186085b32 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:46.496343 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:46.542062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdec47-9403-4460-b2de-bd4186085b32 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:46.625758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd3411-ec43-439d-a9c5-e00912217b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:46.626949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd3411-ec43-439d-a9c5-e00912217b2a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:46.627458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd3411-ec43-439d-a9c5-e00912217b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:46.627937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd3411-ec43-439d-a9c5-e00912217b2a None admin] PATH_INFO: `/v3/users/10cbdc82574341fc8a244bc673d7d584` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:46.636625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd3411-ec43-439d-a9c5-e00912217b2a None admin] RBAC: Authorizing `identity:get_user(user_id=10cbdc82574341fc8a244bc673d7d584)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:46.639553 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd3411-ec43-439d-a9c5-e00912217b2a None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:46.643062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 147/291] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:46 2022] GET /identity/v3/users/10cbdc82574341fc8a244bc673d7d584 => generated 297 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:46.652223 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe433ae-2363-4e34-a080-fcf143289296 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:46.937416 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe433ae-2363-4e34-a080-fcf143289296 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:46.975026 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:47.044739 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8622209-4dfa-4a86-a43c-6d5e7f8e5fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:47.045794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8622209-4dfa-4a86-a43c-6d5e7f8e5fea None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:47.046301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8622209-4dfa-4a86-a43c-6d5e7f8e5fea None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:47.046797 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8622209-4dfa-4a86-a43c-6d5e7f8e5fea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:47.218494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8622209-4dfa-4a86-a43c-6d5e7f8e5fea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:47.221317 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8622209-4dfa-4a86-a43c-6d5e7f8e5fea None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:47.470324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 145/292] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 819 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:47.479022 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd3411-ec43-439d-a9c5-e00912217b2a None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:47.681672 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd3411-ec43-439d-a9c5-e00912217b2a None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:47.770240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce48a23-fdae-4890-9d93-d36c1b887bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:47.771275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce48a23-fdae-4890-9d93-d36c1b887bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:47.771773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce48a23-fdae-4890-9d93-d36c1b887bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:47.772251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce48a23-fdae-4890-9d93-d36c1b887bc5 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:47.774446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce48a23-fdae-4890-9d93-d36c1b887bc5 None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:47.777225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce48a23-fdae-4890-9d93-d36c1b887bc5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:47.786586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 148/293] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:47 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:47.790115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8622209-4dfa-4a86-a43c-6d5e7f8e5fea None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:47.821702 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:48.037590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8622209-4dfa-4a86-a43c-6d5e7f8e5fea None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:48.091050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccdba6-fda1-4806-be2d-00e5fb7f630b None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:48.096835 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccdba6-fda1-4806-be2d-00e5fb7f630b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:48.097366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccdba6-fda1-4806-be2d-00e5fb7f630b None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:48.104227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 149/294] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:48 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:48.114319 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f3c4b-5315-4fb7-95a9-932f9b838104 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:48.114319 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f3c4b-5315-4fb7-95a9-932f9b838104 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:48.114319 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f3c4b-5315-4fb7-95a9-932f9b838104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:48.114781 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a569cb-47f5-47c0-9711-a7adeed57142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:48.115428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a569cb-47f5-47c0-9711-a7adeed57142 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:48.115706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a569cb-47f5-47c0-9711-a7adeed57142 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:48.115976 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a569cb-47f5-47c0-9711-a7adeed57142 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:48.116740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a569cb-47f5-47c0-9711-a7adeed57142 None admin] RBAC: Adding query filter params (role.id=86f185569d134d80841142ae8e9d3f47, scope.project.id=717d2ed44c684799a826e03b360f281b, user.id=10cbdc82574341fc8a244bc673d7d584) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:48.117053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a569cb-47f5-47c0-9711-a7adeed57142 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:48.118538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a569cb-47f5-47c0-9711-a7adeed57142 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:48.126890 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 146/295] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:47 2022] GET /identity/v3/role_assignments?user.id=10cbdc82574341fc8a244bc673d7d584&scope.project.id=717d2ed44c684799a826e03b360f281b&role.id=86f185569d134d80841142ae8e9d3f47 => generated 608 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:48.160177 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-743f3c4b-5315-4fb7-95a9-932f9b838104 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:48.161472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-743f3c4b-5315-4fb7-95a9-932f9b838104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:48.211753 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:48.287674 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 150/296] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:48.291937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd048d9-a5c2-4ea8-907d-079f885e7dc9 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:48.292302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd048d9-a5c2-4ea8-907d-079f885e7dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:48.292591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd048d9-a5c2-4ea8-907d-079f885e7dc9 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:48.294396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 147/297] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:48.302606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afcbcc-bd84-44f4-9e60-2d4e07ebae3e None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:48.302941 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afcbcc-bd84-44f4-9e60-2d4e07ebae3e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:48.303197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afcbcc-bd84-44f4-9e60-2d4e07ebae3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:48.327154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-e4afcbcc-bd84-44f4-9e60-2d4e07ebae3e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:48.328476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e4afcbcc-bd84-44f4-9e60-2d4e07ebae3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:48.437104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 151/298] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:48.715015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd048d9-a5c2-4ea8-907d-079f885e7dc9 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:48.861124 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b0d22-2fe9-4f2d-9bee-b855fcdb2430 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:48.862157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b0d22-2fe9-4f2d-9bee-b855fcdb2430 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:48.862646 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b0d22-2fe9-4f2d-9bee-b855fcdb2430 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:48.865530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 152/299] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:48.872563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97dae98-93b9-4c1c-985a-ee3422a3fbbd None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:48.872860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97dae98-93b9-4c1c-985a-ee3422a3fbbd None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:48.873113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97dae98-93b9-4c1c-985a-ee3422a3fbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:48.894812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:48.911572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-d97dae98-93b9-4c1c-985a-ee3422a3fbbd None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:48.913857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d97dae98-93b9-4c1c-985a-ee3422a3fbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:48.928958 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd048d9-a5c2-4ea8-907d-079f885e7dc9 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:48.971257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbda64-2229-40f2-807d-6b0dafd03831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:48.971810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbda64-2229-40f2-807d-6b0dafd03831 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:48.972109 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbda64-2229-40f2-807d-6b0dafd03831 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:48.972367 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbda64-2229-40f2-807d-6b0dafd03831 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:48.987271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbda64-2229-40f2-807d-6b0dafd03831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-96bbda64-2229-40f2-807d-6b0dafd03831 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:48.995225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:48.997478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:48.998977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:48.998977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:48.998977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:48.998977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:48.998977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:48.998977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:48.998977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:48.998977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:48.998977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:48.998977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:48.998977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:48.998977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:48.998977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 148/300] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:49.001939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbda64-2229-40f2-807d-6b0dafd03831 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:49.055173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 153/301] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:59:49.056871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97dae98-93b9-4c1c-985a-ee3422a3fbbd None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:49.078419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:49.190303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbda64-2229-40f2-807d-6b0dafd03831 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:49.227366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97dae98-93b9-4c1c-985a-ee3422a3fbbd None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:49.261269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2119b-8e29-4792-b2bc-6081bc902354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:49.263342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2119b-8e29-4792-b2bc-6081bc902354 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:49.264280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2119b-8e29-4792-b2bc-6081bc902354 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:49.265097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2119b-8e29-4792-b2bc-6081bc902354 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:49.266814 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2119b-8e29-4792-b2bc-6081bc902354 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:49.267743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2119b-8e29-4792-b2bc-6081bc902354 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:49.270693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2119b-8e29-4792-b2bc-6081bc902354 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:49.274255 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03260432-574c-4710-b11f-5b0c253888fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:49.274934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03260432-574c-4710-b11f-5b0c253888fe None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:49.275253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03260432-574c-4710-b11f-5b0c253888fe None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:49.275531 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03260432-574c-4710-b11f-5b0c253888fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:49.276848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03260432-574c-4710-b11f-5b0c253888fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:49.277818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:49.279554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03260432-574c-4710-b11f-5b0c253888fe None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:49.281953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 154/302] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:59:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:49.288363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03260432-574c-4710-b11f-5b0c253888fe None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:49.294440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 149/303] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:49.301130 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2119b-8e29-4792-b2bc-6081bc902354 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:49.412624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:49.447399 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03260432-574c-4710-b11f-5b0c253888fe None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:49.473507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2119b-8e29-4792-b2bc-6081bc902354 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:49.499698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03fdba-5055-47ce-b325-cb351e8d6ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:49.500544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03fdba-5055-47ce-b325-cb351e8d6ba6 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:49.500824 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03fdba-5055-47ce-b325-cb351e8d6ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:49.501095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03fdba-5055-47ce-b325-cb351e8d6ba6 None admin] PATH_INFO: `/v3/groups` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:49.502370 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03fdba-5055-47ce-b325-cb351e8d6ba6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:49.504494 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03fdba-5055-47ce-b325-cb351e8d6ba6 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:49.531147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 155/304] 10.222.0.61 () {60 vars in 1134 bytes} [Wed Sep 7 12:59:49 2022] POST /identity/v3/groups => generated 229 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:59:49.531147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60144dbf-eaa2-4cb0-88a2-0aaa4690e1ff None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:49.531147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60144dbf-eaa2-4cb0-88a2-0aaa4690e1ff None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:49.531147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60144dbf-eaa2-4cb0-88a2-0aaa4690e1ff None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:49.535573 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 156/305] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:49 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:49.537996 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b07822-deab-45d2-8bac-7d7749307347 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:49.538336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b07822-deab-45d2-8bac-7d7749307347 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:49.538601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b07822-deab-45d2-8bac-7d7749307347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:49.546837 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e152e-d60e-47a0-8cba-2853a11eb67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:49.548033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e152e-d60e-47a0-8cba-2853a11eb67b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:49.548590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e152e-d60e-47a0-8cba-2853a11eb67b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:49.549114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e152e-d60e-47a0-8cba-2853a11eb67b None admin] PATH_INFO: `/v3/users/5227e2bb2566404b938f297538a80b8e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:49.564426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-d9b07822-deab-45d2-8bac-7d7749307347 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:49.566430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d9b07822-deab-45d2-8bac-7d7749307347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:49.578710 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e152e-d60e-47a0-8cba-2853a11eb67b None admin] RBAC: Authorizing `identity:get_user(user_id=5227e2bb2566404b938f297538a80b8e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:49.581541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e152e-d60e-47a0-8cba-2853a11eb67b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:49.585034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 150/306] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:49 2022] GET /identity/v3/users/5227e2bb2566404b938f297538a80b8e => generated 293 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:49.586272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e152e-d60e-47a0-8cba-2853a11eb67b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:49.694122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 157/307] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:49.695819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b07822-deab-45d2-8bac-7d7749307347 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:49.697083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:49.810790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e152e-d60e-47a0-8cba-2853a11eb67b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:49.822056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:49.877008 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b07822-deab-45d2-8bac-7d7749307347 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:49.891295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6825ce-5917-4a53-a929-9d6399e80f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:49.893121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6825ce-5917-4a53-a929-9d6399e80f2b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:49.893611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6825ce-5917-4a53-a929-9d6399e80f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:49.894325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6825ce-5917-4a53-a929-9d6399e80f2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:49.917039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6825ce-5917-4a53-a929-9d6399e80f2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:49.925314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d3618-cf88-46a2-9386-f260f72ddfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:49.925995 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d3618-cf88-46a2-9386-f260f72ddfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:49.926280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d3618-cf88-46a2-9386-f260f72ddfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:49.926556 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d3618-cf88-46a2-9386-f260f72ddfe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-0f6825ce-5917-4a53-a929-9d6399e80f2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:49.927146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:49.928811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:49.930423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:49.930423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:49.930423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:49.930423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:49.930423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:49.930423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:49.930423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:49.930423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:49.930423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:49.930423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:49.930423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:49.930423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:49.932241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d3618-cf88-46a2-9386-f260f72ddfe2 None admin] RBAC: Adding query filter params (role.id=4b22e77ca0b449f3b83e08e32d811efb, user.id=5227e2bb2566404b938f297538a80b8e, scope.system=all) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:49.932241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d3618-cf88-46a2-9386-f260f72ddfe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:49.933118 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 151/308] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:49.933241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d3618-cf88-46a2-9386-f260f72ddfe2 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:49.941354 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6825ce-5917-4a53-a929-9d6399e80f2b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:49.946952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 158/309] 10.222.0.61 () {56 vars in 1308 bytes} [Wed Sep 7 12:59:49 2022] GET /identity/v3/role_assignments?user.id=5227e2bb2566404b938f297538a80b8e&scope.system=all&role.id=4b22e77ca0b449f3b83e08e32d811efb => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:50.006864 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d3618-cf88-46a2-9386-f260f72ddfe2 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:50.136264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6825ce-5917-4a53-a929-9d6399e80f2b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:50.160744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:50.183620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18540b-2b61-4ada-bcd3-8ba7d1f98e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:50.184959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18540b-2b61-4ada-bcd3-8ba7d1f98e72 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:50.185266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18540b-2b61-4ada-bcd3-8ba7d1f98e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:50.185521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18540b-2b61-4ada-bcd3-8ba7d1f98e72 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:50.186310 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18540b-2b61-4ada-bcd3-8ba7d1f98e72 None admin] RBAC: Adding query filter params (name=member) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:50.186622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18540b-2b61-4ada-bcd3-8ba7d1f98e72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:50.188115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18540b-2b61-4ada-bcd3-8ba7d1f98e72 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:50.204535 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 152/310] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:50.206114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18540b-2b61-4ada-bcd3-8ba7d1f98e72 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:50.256205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d3618-cf88-46a2-9386-f260f72ddfe2 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:50.330887 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9048c-25af-4cdb-abc5-d8f20c8f293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:50.332463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9048c-25af-4cdb-abc5-d8f20c8f293b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:50.333183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9048c-25af-4cdb-abc5-d8f20c8f293b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:50.333909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9048c-25af-4cdb-abc5-d8f20c8f293b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:50.335389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:50.355300 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a9048c-25af-4cdb-abc5-d8f20c8f293b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-93a9048c-25af-4cdb-abc5-d8f20c8f293b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:50.370339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:50.372721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:50.374279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:50.374279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:50.374279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:50.374279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:50.374279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:50.374279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:50.374279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:50.374279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:50.374279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:50.374279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:50.374279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:50.374279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:50.374920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 159/311] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:59:50.384169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9048c-25af-4cdb-abc5-d8f20c8f293b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:50.386715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18540b-2b61-4ada-bcd3-8ba7d1f98e72 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:50.438165 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07a3cb-1f7e-4b3f-9f1b-a3c147eb2e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:50.440041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07a3cb-1f7e-4b3f-9f1b-a3c147eb2e1e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:50.440662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07a3cb-1f7e-4b3f-9f1b-a3c147eb2e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:50.441498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07a3cb-1f7e-4b3f-9f1b-a3c147eb2e1e None admin] PATH_INFO: `/v3/users/16ae927e7ced46c79a8a48f9d7dc0ab9` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:50.443516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07a3cb-1f7e-4b3f-9f1b-a3c147eb2e1e None admin] RBAC: Authorizing `identity:get_user(user_id=16ae927e7ced46c79a8a48f9d7dc0ab9)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:50.446008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07a3cb-1f7e-4b3f-9f1b-a3c147eb2e1e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:50.453867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 153/312] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:50 2022] GET /identity/v3/users/16ae927e7ced46c79a8a48f9d7dc0ab9 => generated 293 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:50.457333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07a3cb-1f7e-4b3f-9f1b-a3c147eb2e1e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:50.537791 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:50.563622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a9048c-25af-4cdb-abc5-d8f20c8f293b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:50.612988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8fe4d-948e-456b-aaa9-1bb63b661d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:50.614003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8fe4d-948e-456b-aaa9-1bb63b661d4b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:50.614478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8fe4d-948e-456b-aaa9-1bb63b661d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:50.614938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8fe4d-948e-456b-aaa9-1bb63b661d4b None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:50.616784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8fe4d-948e-456b-aaa9-1bb63b661d4b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:50.617354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8fe4d-948e-456b-aaa9-1bb63b661d4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:50.619736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8fe4d-948e-456b-aaa9-1bb63b661d4b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:50.633432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 160/313] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:50.643398 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8fe4d-948e-456b-aaa9-1bb63b661d4b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:50.644492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07a3cb-1f7e-4b3f-9f1b-a3c147eb2e1e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:50.695619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc7333-ac4e-4e6e-b40b-34df042200e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:50.696688 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc7333-ac4e-4e6e-b40b-34df042200e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:50.697506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc7333-ac4e-4e6e-b40b-34df042200e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:50.697990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc7333-ac4e-4e6e-b40b-34df042200e0 None admin] PATH_INFO: `/v3/system/users/16ae927e7ced46c79a8a48f9d7dc0ab9/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:50.700939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc7333-ac4e-4e6e-b40b-34df042200e0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=16ae927e7ced46c79a8a48f9d7dc0ab9, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:50.703693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc7333-ac4e-4e6e-b40b-34df042200e0 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:50.723915 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 154/314] 10.222.0.61 () {58 vars in 1334 bytes} [Wed Sep 7 12:59:50 2022] PUT /identity/v3/system/users/16ae927e7ced46c79a8a48f9d7dc0ab9/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:59:50.743816 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:50.853004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8fe4d-948e-456b-aaa9-1bb63b661d4b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:50.919401 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d32abe-2f23-49e5-82d7-2fa824f52e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:50.920452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d32abe-2f23-49e5-82d7-2fa824f52e15 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:50.920955 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d32abe-2f23-49e5-82d7-2fa824f52e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:50.921446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d32abe-2f23-49e5-82d7-2fa824f52e15 None admin] PATH_INFO: `/v3/users/1a93a8336a85455e97e9455660d180c1` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:50.923785 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d32abe-2f23-49e5-82d7-2fa824f52e15 None admin] RBAC: Authorizing `identity:get_user(user_id=1a93a8336a85455e97e9455660d180c1)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:50.926456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d32abe-2f23-49e5-82d7-2fa824f52e15 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:50.929865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 161/315] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:50 2022] GET /identity/v3/users/1a93a8336a85455e97e9455660d180c1 => generated 297 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:50.940186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc7333-ac4e-4e6e-b40b-34df042200e0 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:51.127269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc7333-ac4e-4e6e-b40b-34df042200e0 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:51.140744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:51.167916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8dd77a-1242-42ba-8460-571f85c5f54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:51.169473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dd77a-1242-42ba-8460-571f85c5f54a None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:51.169936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dd77a-1242-42ba-8460-571f85c5f54a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:51.170389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dd77a-1242-42ba-8460-571f85c5f54a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:51.254183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8dd77a-1242-42ba-8460-571f85c5f54a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:51.255721 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8dd77a-1242-42ba-8460-571f85c5f54a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:51.373601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 155/316] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:51.379863 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d32abe-2f23-49e5-82d7-2fa824f52e15 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:51.517656 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d32abe-2f23-49e5-82d7-2fa824f52e15 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:51.539187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:51.562653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869c7bd-8f9e-4d2c-85db-2e82421077c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:51.563652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869c7bd-8f9e-4d2c-85db-2e82421077c2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:51.564116 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869c7bd-8f9e-4d2c-85db-2e82421077c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:51.564572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869c7bd-8f9e-4d2c-85db-2e82421077c2 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:51.566037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869c7bd-8f9e-4d2c-85db-2e82421077c2 None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:51.567817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869c7bd-8f9e-4d2c-85db-2e82421077c2 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:51.570129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 162/317] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:51 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:51.576712 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8dd77a-1242-42ba-8460-571f85c5f54a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:51.591492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:51.696121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8dd77a-1242-42ba-8460-571f85c5f54a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:51.738601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc3601-6df7-4753-a6a5-c875a76ac73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:51.739362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3601-6df7-4753-a6a5-c875a76ac73c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:51.739652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3601-6df7-4753-a6a5-c875a76ac73c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:51.739973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc3601-6df7-4753-a6a5-c875a76ac73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:51.740744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc3601-6df7-4753-a6a5-c875a76ac73c None admin] RBAC: Adding query filter params (role.id=4b22e77ca0b449f3b83e08e32d811efb, scope.project.id=717d2ed44c684799a826e03b360f281b, user.id=1a93a8336a85455e97e9455660d180c1) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:51.741061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc3601-6df7-4753-a6a5-c875a76ac73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:51.742571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc3601-6df7-4753-a6a5-c875a76ac73c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:51.750390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 156/318] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 12:59:51 2022] GET /identity/v3/role_assignments?user.id=1a93a8336a85455e97e9455660d180c1&scope.project.id=717d2ed44c684799a826e03b360f281b&role.id=4b22e77ca0b449f3b83e08e32d811efb => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:52.394558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448843e2-78b7-4657-8977-ee98a6c62725 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:52.395081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448843e2-78b7-4657-8977-ee98a6c62725 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:52.395081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448843e2-78b7-4657-8977-ee98a6c62725 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:52.396721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 163/319] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:52.402187 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a4113-f780-4cbe-91a2-25ed59f1f89c None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:52.402764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a4113-f780-4cbe-91a2-25ed59f1f89c None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:52.402764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a4113-f780-4cbe-91a2-25ed59f1f89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:52.424951 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-999a4113-f780-4cbe-91a2-25ed59f1f89c None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:52.426037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-999a4113-f780-4cbe-91a2-25ed59f1f89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:52.519727 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 157/320] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:52.754113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a9589-5312-4fb2-9901-dbf130fa1ff4 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:52.754356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a9589-5312-4fb2-9901-dbf130fa1ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:52.754765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a9589-5312-4fb2-9901-dbf130fa1ff4 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:52.756625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 164/321] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:52.769172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f32e4-1e5c-4b0c-9fb4-dda3af7185d2 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:52.769644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f32e4-1e5c-4b0c-9fb4-dda3af7185d2 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:52.770120 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f32e4-1e5c-4b0c-9fb4-dda3af7185d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:52.783395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:52.800358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a9589-5312-4fb2-9901-dbf130fa1ff4 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:52.803482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-926f32e4-1e5c-4b0c-9fb4-dda3af7185d2 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:52.804522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-926f32e4-1e5c-4b0c-9fb4-dda3af7185d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:52.907146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 158/322] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:52.987318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a9589-5312-4fb2-9901-dbf130fa1ff4 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:53.052108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e9664-b4c0-4b3a-b1c7-4f2cef221505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:53.053192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e9664-b4c0-4b3a-b1c7-4f2cef221505 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:53.053666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e9664-b4c0-4b3a-b1c7-4f2cef221505 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:53.054150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e9664-b4c0-4b3a-b1c7-4f2cef221505 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:53.079385 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e9664-b4c0-4b3a-b1c7-4f2cef221505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-023e9664-b4c0-4b3a-b1c7-4f2cef221505 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:53.095780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:53.097607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:53.099758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:53.099758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:53.099758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:53.099758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:53.099758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:53.099758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:53.099758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:53.099758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:53.099758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:53.099758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:53.099758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:53.099758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:53.099758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 165/323] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:59:53.102317 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f32e4-1e5c-4b0c-9fb4-dda3af7185d2 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:53.313683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f32e4-1e5c-4b0c-9fb4-dda3af7185d2 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:53.375537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e9664-b4c0-4b3a-b1c7-4f2cef221505 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:53.381256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65c141-97f3-4c6f-8a2d-92554aefdb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:53.382232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65c141-97f3-4c6f-8a2d-92554aefdb7f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:53.382694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65c141-97f3-4c6f-8a2d-92554aefdb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:53.383236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65c141-97f3-4c6f-8a2d-92554aefdb7f None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:53.384032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65c141-97f3-4c6f-8a2d-92554aefdb7f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:53.384484 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65c141-97f3-4c6f-8a2d-92554aefdb7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:53.386271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65c141-97f3-4c6f-8a2d-92554aefdb7f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:53.388490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:53.401446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 159/324] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:53.410825 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65c141-97f3-4c6f-8a2d-92554aefdb7f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:53.449993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:53.517662 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e9664-b4c0-4b3a-b1c7-4f2cef221505 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:53.567473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65c141-97f3-4c6f-8a2d-92554aefdb7f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:53.581581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f0dfb-24dd-42d6-adfd-98968dd95b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:53.582658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f0dfb-24dd-42d6-adfd-98968dd95b30 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:53.583161 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f0dfb-24dd-42d6-adfd-98968dd95b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:53.583591 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f0dfb-24dd-42d6-adfd-98968dd95b30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:53.585831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f0dfb-24dd-42d6-adfd-98968dd95b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:53.587573 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f0dfb-24dd-42d6-adfd-98968dd95b30 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:53.589291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 166/325] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 12:59:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:53.590828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5207c16-4784-4398-987a-cae8639597f6 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:53.591284 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5207c16-4784-4398-987a-cae8639597f6 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:53.591495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5207c16-4784-4398-987a-cae8639597f6 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:53.593139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 167/326] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:53.598437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5207c16-4784-4398-987a-cae8639597f6 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:53.613550 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d791c78-2586-403f-b0e9-aeab88b42bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:53.614215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d791c78-2586-403f-b0e9-aeab88b42bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:53.614743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d791c78-2586-403f-b0e9-aeab88b42bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:53.615024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d791c78-2586-403f-b0e9-aeab88b42bc4 None admin] PATH_INFO: `/v3/users/5227e2bb2566404b938f297538a80b8e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:53.616741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d791c78-2586-403f-b0e9-aeab88b42bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=5227e2bb2566404b938f297538a80b8e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:53.618404 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d791c78-2586-403f-b0e9-aeab88b42bc4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:53.620175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 160/327] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:53 2022] GET /identity/v3/users/5227e2bb2566404b938f297538a80b8e => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:53.623661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a75a26-63cb-40ce-a295-07ef7d74f349 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:53.624574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a75a26-63cb-40ce-a295-07ef7d74f349 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:53.625016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a75a26-63cb-40ce-a295-07ef7d74f349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:53.653275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-e5a75a26-63cb-40ce-a295-07ef7d74f349 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:53.654512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e5a75a26-63cb-40ce-a295-07ef7d74f349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:53.724241 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:53.734079 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 161/328] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:53.735341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a75a26-63cb-40ce-a295-07ef7d74f349 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:53.778471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5207c16-4784-4398-987a-cae8639597f6 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:53.805933 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:53.845367 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbac6b-a469-4e45-9cb8-fabfb4caf434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:53.846684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbac6b-a469-4e45-9cb8-fabfb4caf434 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:53.846824 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbac6b-a469-4e45-9cb8-fabfb4caf434 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:53.847282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbac6b-a469-4e45-9cb8-fabfb4caf434 None admin] PATH_INFO: `/v3/groups` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:53.848451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbac6b-a469-4e45-9cb8-fabfb4caf434 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:53.851247 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbac6b-a469-4e45-9cb8-fabfb4caf434 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:53.861527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a75a26-63cb-40ce-a295-07ef7d74f349 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:53.887293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 168/329] 10.222.0.61 () {60 vars in 1134 bytes} [Wed Sep 7 12:59:53 2022] POST /identity/v3/groups => generated 226 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 12:59:53.913704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb483f-71c4-4ec6-9fe4-33a6fedb38f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:53.914458 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb483f-71c4-4ec6-9fe4-33a6fedb38f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:53.914747 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb483f-71c4-4ec6-9fe4-33a6fedb38f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:53.915083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb483f-71c4-4ec6-9fe4-33a6fedb38f9 None admin] PATH_INFO: `/v3/system/users/5227e2bb2566404b938f297538a80b8e/roles/4b22e77ca0b449f3b83e08e32d811efb` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:53.916799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb483f-71c4-4ec6-9fe4-33a6fedb38f9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5227e2bb2566404b938f297538a80b8e, role_id=4b22e77ca0b449f3b83e08e32d811efb)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:53.918771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb483f-71c4-4ec6-9fe4-33a6fedb38f9 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:53.929230 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 162/330] 10.222.0.61 () {58 vars in 1334 bytes} [Wed Sep 7 12:59:53 2022] PUT /identity/v3/system/users/5227e2bb2566404b938f297538a80b8e/roles/4b22e77ca0b449f3b83e08e32d811efb => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 12:59:54.114379 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbac6b-a469-4e45-9cb8-fabfb4caf434 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:54.251459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbac6b-a469-4e45-9cb8-fabfb4caf434 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:54.299245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4bcbf-7f2d-4dbf-9cd3-cb6e26db26d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:54.299926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4bcbf-7f2d-4dbf-9cd3-cb6e26db26d2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:54.300204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4bcbf-7f2d-4dbf-9cd3-cb6e26db26d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:54.300466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4bcbf-7f2d-4dbf-9cd3-cb6e26db26d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:54.314180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4bcbf-7f2d-4dbf-9cd3-cb6e26db26d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-18c4bcbf-7f2d-4dbf-9cd3-cb6e26db26d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:54.319908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:54.321060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:54.322432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:54.322432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:54.322432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:54.322432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:54.322432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:54.322432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:54.322432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:54.322432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:54.322432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:54.322432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:54.322432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:54.322432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:54.322432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 169/331] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:59:54.327884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb483f-71c4-4ec6-9fe4-33a6fedb38f9 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:54.411900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:54.492888 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb483f-71c4-4ec6-9fe4-33a6fedb38f9 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:54.565315 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5b0e6-9461-48df-b461-230a88442dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:54.566366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5b0e6-9461-48df-b461-230a88442dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:54.566852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5b0e6-9461-48df-b461-230a88442dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:54.567415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5b0e6-9461-48df-b461-230a88442dc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:54.568653 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5b0e6-9461-48df-b461-230a88442dc9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:54.569194 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5b0e6-9461-48df-b461-230a88442dc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:54.571948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5b0e6-9461-48df-b461-230a88442dc9 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:54.589890 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 163/332] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:54.599333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4bcbf-7f2d-4dbf-9cd3-cb6e26db26d2 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:54.695027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:54.835974 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4bcbf-7f2d-4dbf-9cd3-cb6e26db26d2 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:54.913242 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e467e3c-1f2a-40a2-8422-4698a6e93842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:54.913943 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e467e3c-1f2a-40a2-8422-4698a6e93842 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:54.914418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e467e3c-1f2a-40a2-8422-4698a6e93842 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:54.914885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e467e3c-1f2a-40a2-8422-4698a6e93842 None admin] PATH_INFO: `/v3/users/16ae927e7ced46c79a8a48f9d7dc0ab9` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:54.917143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e467e3c-1f2a-40a2-8422-4698a6e93842 None admin] RBAC: Authorizing `identity:get_user(user_id=16ae927e7ced46c79a8a48f9d7dc0ab9)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:54.920624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e467e3c-1f2a-40a2-8422-4698a6e93842 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:54.923786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 170/333] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:54 2022] GET /identity/v3/users/16ae927e7ced46c79a8a48f9d7dc0ab9 => generated 293 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:54.932652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5b0e6-9461-48df-b461-230a88442dc9 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:55.139680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5b0e6-9461-48df-b461-230a88442dc9 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:55.182401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:55.217369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf341b-7a5a-4325-b20f-21f0f6204504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:55.218371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf341b-7a5a-4325-b20f-21f0f6204504 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:55.218846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf341b-7a5a-4325-b20f-21f0f6204504 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:55.219350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf341b-7a5a-4325-b20f-21f0f6204504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:55.220634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf341b-7a5a-4325-b20f-21f0f6204504 None admin] RBAC: Adding query filter params (role.id=86f185569d134d80841142ae8e9d3f47, user.id=16ae927e7ced46c79a8a48f9d7dc0ab9, scope.system=all) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:55.221173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf341b-7a5a-4325-b20f-21f0f6204504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:55.224003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf341b-7a5a-4325-b20f-21f0f6204504 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:55.248833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 164/334] 10.222.0.61 () {56 vars in 1308 bytes} [Wed Sep 7 12:59:54 2022] GET /identity/v3/role_assignments?user.id=16ae927e7ced46c79a8a48f9d7dc0ab9&scope.system=all&role.id=86f185569d134d80841142ae8e9d3f47 => generated 510 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:57.028292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318697d4-6cf8-4ca3-a80c-db39368e5d94 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:57.029050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318697d4-6cf8-4ca3-a80c-db39368e5d94 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:57.029277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318697d4-6cf8-4ca3-a80c-db39368e5d94 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:57.032206 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 171/335] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:57.039357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef2198-c638-489d-8cbd-4d1e985895bd None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:57.040950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef2198-c638-489d-8cbd-4d1e985895bd None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:57.041715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef2198-c638-489d-8cbd-4d1e985895bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:57.080270 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b9ef2198-c638-489d-8cbd-4d1e985895bd None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:57.083125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b9ef2198-c638-489d-8cbd-4d1e985895bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:57.239742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 165/336] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:57.823577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318697d4-6cf8-4ca3-a80c-db39368e5d94 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:57.864504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fea6e-22b0-43c0-9043-1dad8e220e47 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:57.864784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fea6e-22b0-43c0-9043-1dad8e220e47 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:57.865121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fea6e-22b0-43c0-9043-1dad8e220e47 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:57.866771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 166/337] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 12:59:57.873113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3b698-47e3-4d69-85da-b08ea9076613 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:57.873385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3b698-47e3-4d69-85da-b08ea9076613 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:57.873668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3b698-47e3-4d69-85da-b08ea9076613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:57.912276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-64c3b698-47e3-4d69-85da-b08ea9076613 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:57.913513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-64c3b698-47e3-4d69-85da-b08ea9076613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:58.119755 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 167/338] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 12:59:58.124059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a82b17-c80a-486f-ab75-8203f8818219 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:58.125085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a82b17-c80a-486f-ab75-8203f8818219 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:58.125821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a82b17-c80a-486f-ab75-8203f8818219 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:58.129696 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 168/339] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:58.142569 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c7dda-8676-4407-ae3c-50395c7b057e None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:58.146707 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c7dda-8676-4407-ae3c-50395c7b057e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:58.149031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c7dda-8676-4407-ae3c-50395c7b057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:58.177516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318697d4-6cf8-4ca3-a80c-db39368e5d94 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:58.187652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:58.226371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-5f3c7dda-8676-4407-ae3c-50395c7b057e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:58.228519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5f3c7dda-8676-4407-ae3c-50395c7b057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:58.302846 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6903c9f-afb7-4654-860f-e6bba28629fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:58.304003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6903c9f-afb7-4654-860f-e6bba28629fd None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:58.305460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6903c9f-afb7-4654-860f-e6bba28629fd None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:58.305968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6903c9f-afb7-4654-860f-e6bba28629fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:58.344832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6903c9f-afb7-4654-860f-e6bba28629fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-a6903c9f-afb7-4654-860f-e6bba28629fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:58.356150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:58.357956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:58.359574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:58.359574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:58.359574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:58.359574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:58.359574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:58.359574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:58.359574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:58.359574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:58.359574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:58.359574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:58.359574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 12:59:58.359574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 12:59:58.360756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:58.366168 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 172/340] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 542 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:58.375286 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6903c9f-afb7-4654-860f-e6bba28629fd None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:58.473047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 169/341] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:58.546693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c7dda-8676-4407-ae3c-50395c7b057e None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:58.602444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6903c9f-afb7-4654-860f-e6bba28629fd None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:58.680715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c5dd0-ccf5-4861-adaf-00be3d27ad42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:58.681351 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c5dd0-ccf5-4861-adaf-00be3d27ad42 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:58.681626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c5dd0-ccf5-4861-adaf-00be3d27ad42 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:58.681885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c5dd0-ccf5-4861-adaf-00be3d27ad42 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:58.682572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c5dd0-ccf5-4861-adaf-00be3d27ad42 None admin] RBAC: Adding query filter params (name=member) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:58.682870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c5dd0-ccf5-4861-adaf-00be3d27ad42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:58.684928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c5dd0-ccf5-4861-adaf-00be3d27ad42 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:58.704606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 173/342] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:58.716045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c5dd0-ccf5-4861-adaf-00be3d27ad42 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:58.748925 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c7dda-8676-4407-ae3c-50395c7b057e None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:58.787712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:58.846436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9087446-53ed-465c-b3e5-9de0def17f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:58.847242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9087446-53ed-465c-b3e5-9de0def17f3c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:58.847555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9087446-53ed-465c-b3e5-9de0def17f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:58.847819 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9087446-53ed-465c-b3e5-9de0def17f3c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:58.876357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9087446-53ed-465c-b3e5-9de0def17f3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-f9087446-53ed-465c-b3e5-9de0def17f3c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:58.882527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:58.884703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:58.886506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:58.886506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:58.886506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:58.886506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:58.886506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:58.886506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:58.886506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:58.886506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:58.886506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:58.886506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:58.886506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 12:59:58.886506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:58.886506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 170/343] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 12:59:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 12:59:58.896521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9087446-53ed-465c-b3e5-9de0def17f3c None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:58.959512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:59.060070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c5dd0-ccf5-4861-adaf-00be3d27ad42 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:59.164899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4270c2-d711-44b7-9505-807b69714f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:59.166504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4270c2-d711-44b7-9505-807b69714f32 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:59.167374 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4270c2-d711-44b7-9505-807b69714f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:59.168709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4270c2-d711-44b7-9505-807b69714f32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:59.180151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9087446-53ed-465c-b3e5-9de0def17f3c None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:59.309543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca0c95-f675-4f12-9b5c-b3a251eac5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:59.311395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca0c95-f675-4f12-9b5c-b3a251eac5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:59.312266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca0c95-f675-4f12-9b5c-b3a251eac5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:59.313052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca0c95-f675-4f12-9b5c-b3a251eac5c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:59.314716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca0c95-f675-4f12-9b5c-b3a251eac5c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 12:59:59.316063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca0c95-f675-4f12-9b5c-b3a251eac5c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:59.321319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca0c95-f675-4f12-9b5c-b3a251eac5c2 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:59.364738 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 171/344] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 12:59:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 12:59:59.366898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca0c95-f675-4f12-9b5c-b3a251eac5c2 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:59.376389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4270c2-d711-44b7-9505-807b69714f32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:59.379500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4270c2-d711-44b7-9505-807b69714f32 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:59.443897 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:59.555149 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 174/345] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 12:59:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 834 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 12:59:59.555149 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4270c2-d711-44b7-9505-807b69714f32 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:59.596113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:59.613360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca0c95-f675-4f12-9b5c-b3a251eac5c2 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:59.674343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a5165-b267-438f-8e97-187ab870d7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:59.675004 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a5165-b267-438f-8e97-187ab870d7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:59.675342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a5165-b267-438f-8e97-187ab870d7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:59.675615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a5165-b267-438f-8e97-187ab870d7c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:59.690302 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4270c2-d711-44b7-9505-807b69714f32 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:59.691606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a5165-b267-438f-8e97-187ab870d7c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-f33a5165-b267-438f-8e97-187ab870d7c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 12:59:59.697467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 12:59:59.698432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 12:59:59.699342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 12:59:59.699342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 12:59:59.699342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 12:59:59.699342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 12:59:59.699342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 12:59:59.699342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 12:59:59.699342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 12:59:59.699342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 12:59:59.699342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 12:59:59.699342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 12:59:59.699342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 12:59:59.699342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 12:59:59.699342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 172/346] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 12:59:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 12:59:59.700028 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334abd47-5561-4822-ad33-2e99d0e43dc3 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:59.700605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334abd47-5561-4822-ad33-2e99d0e43dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:59.700605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334abd47-5561-4822-ad33-2e99d0e43dc3 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:59.702061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 173/347] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:59.702886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334abd47-5561-4822-ad33-2e99d0e43dc3 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:59.749735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 12:59:59.770220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27ff2f-9b01-4109-a800-9b493b2bf12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:59.771289 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27ff2f-9b01-4109-a800-9b493b2bf12b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:59.771734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27ff2f-9b01-4109-a800-9b493b2bf12b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:59.772162 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27ff2f-9b01-4109-a800-9b493b2bf12b None admin] PATH_INFO: `/v3/users/5227e2bb2566404b938f297538a80b8e` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:59.774351 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27ff2f-9b01-4109-a800-9b493b2bf12b None admin] RBAC: Authorizing `identity:get_user(user_id=5227e2bb2566404b938f297538a80b8e)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 12:59:59.777641 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27ff2f-9b01-4109-a800-9b493b2bf12b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 12:59:59.781015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 175/348] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 12:59:59 2022] GET /identity/v3/users/5227e2bb2566404b938f297538a80b8e => generated 293 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 12:59:59.784221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83739f33-e6db-4331-920d-c9b2f2755e58 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:59.784586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83739f33-e6db-4331-920d-c9b2f2755e58 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:59.785075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83739f33-e6db-4331-920d-c9b2f2755e58 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:59.787703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 176/349] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 12:59:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 12:59:59.790092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c2f78-8cc5-4257-88f2-02bca7c6d30a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:59.790587 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c2f78-8cc5-4257-88f2-02bca7c6d30a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:59.791079 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c2f78-8cc5-4257-88f2-02bca7c6d30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 12:59:59.817985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-bb8c2f78-8cc5-4257-88f2-02bca7c6d30a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 12:59:59.819586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-bb8c2f78-8cc5-4257-88f2-02bca7c6d30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 12:59:59.916519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334abd47-5561-4822-ad33-2e99d0e43dc3 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 12:59:59.940490 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 177/350] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 12:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 12:59:59.945664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c2f78-8cc5-4257-88f2-02bca7c6d30a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 12:59:59.996570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb99f0-f335-4b65-ae60-ccf75500c0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 12:59:59.997766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb99f0-f335-4b65-ae60-ccf75500c0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 12:59:59.998359 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb99f0-f335-4b65-ae60-ccf75500c0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 12:59:59.998923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb99f0-f335-4b65-ae60-ccf75500c0e3 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:00.001571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb99f0-f335-4b65-ae60-ccf75500c0e3 None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:00.004762 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb99f0-f335-4b65-ae60-ccf75500c0e3 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:00.009860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 174/351] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 12:59:59 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:00.012231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb99f0-f335-4b65-ae60-ccf75500c0e3 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:00.051964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:00.118261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c2f78-8cc5-4257-88f2-02bca7c6d30a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:00.184233 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:00.195835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb99f0-f335-4b65-ae60-ccf75500c0e3 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:00.200283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb4751-41cf-453e-b9a0-690d30631bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:00.201500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb4751-41cf-453e-b9a0-690d30631bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:00.202092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb4751-41cf-453e-b9a0-690d30631bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:00.202665 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb4751-41cf-453e-b9a0-690d30631bd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:00.204062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb4751-41cf-453e-b9a0-690d30631bd1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:00.204697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb4751-41cf-453e-b9a0-690d30631bd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:00.207570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb4751-41cf-453e-b9a0-690d30631bd1 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:00.226768 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 178/352] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 12:59:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:00.233907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144d83f-0a7e-4adf-8e71-4c7950c9f1e0 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:00.234396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144d83f-0a7e-4adf-8e71-4c7950c9f1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:00.234893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144d83f-0a7e-4adf-8e71-4c7950c9f1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:00.242456 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889830c0-d04e-4a3b-b8fa-f52d21b02f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:00.243093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889830c0-d04e-4a3b-b8fa-f52d21b02f6c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:00.243417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889830c0-d04e-4a3b-b8fa-f52d21b02f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:00.243697 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889830c0-d04e-4a3b-b8fa-f52d21b02f6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:00.244636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889830c0-d04e-4a3b-b8fa-f52d21b02f6c None admin] RBAC: Adding query filter params (role.id=4b22e77ca0b449f3b83e08e32d811efb, user.id=5227e2bb2566404b938f297538a80b8e, scope.system=all) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:00.245129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889830c0-d04e-4a3b-b8fa-f52d21b02f6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:00.247264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889830c0-d04e-4a3b-b8fa-f52d21b02f6c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:00.264469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 175/353] 10.222.0.61 () {56 vars in 1308 bytes} [Wed Sep 7 13:00:00 2022] GET /identity/v3/role_assignments?user.id=5227e2bb2566404b938f297538a80b8e&scope.system=all&role.id=4b22e77ca0b449f3b83e08e32d811efb => generated 510 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:00.265367 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889830c0-d04e-4a3b-b8fa-f52d21b02f6c None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:00.277935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-a144d83f-0a7e-4adf-8e71-4c7950c9f1e0 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:00.280198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a144d83f-0a7e-4adf-8e71-4c7950c9f1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:00.379723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:00.424481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 179/354] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:00.426059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889830c0-d04e-4a3b-b8fa-f52d21b02f6c None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:00.426209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144d83f-0a7e-4adf-8e71-4c7950c9f1e0 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:00.476799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06ed42-3cd4-405f-8200-776ef184b77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:00.477749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06ed42-3cd4-405f-8200-776ef184b77b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:00.478667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06ed42-3cd4-405f-8200-776ef184b77b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:00.479164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06ed42-3cd4-405f-8200-776ef184b77b None admin] PATH_INFO: `/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:00.488461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06ed42-3cd4-405f-8200-776ef184b77b None admin] RBAC: Authorizing `identity:get_group(group_id=0be5c6a5a91743e3a1e5a7676f39026e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:00.491092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06ed42-3cd4-405f-8200-776ef184b77b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:00.493770 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 176/355] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:00 2022] GET /identity/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:00.500906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06ed42-3cd4-405f-8200-776ef184b77b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:00.635592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06ed42-3cd4-405f-8200-776ef184b77b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:00.681283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144d83f-0a7e-4adf-8e71-4c7950c9f1e0 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:00.685043 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c3281-3441-426c-a4d8-966602ebba49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:00.685722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c3281-3441-426c-a4d8-966602ebba49 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:00.686037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c3281-3441-426c-a4d8-966602ebba49 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:00.686340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c3281-3441-426c-a4d8-966602ebba49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:00.687143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c3281-3441-426c-a4d8-966602ebba49 None admin] RBAC: Adding query filter params (group.id=0be5c6a5a91743e3a1e5a7676f39026e, role.id=86f185569d134d80841142ae8e9d3f47, scope.project.id=81be097728134d76957fdb5755382cc2) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:00.687451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c3281-3441-426c-a4d8-966602ebba49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:00.688955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c3281-3441-426c-a4d8-966602ebba49 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:00.703583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 177/356] 10.222.0.61 () {56 vars in 1376 bytes} [Wed Sep 7 13:00:00 2022] GET /identity/v3/role_assignments?group.id=0be5c6a5a91743e3a1e5a7676f39026e&scope.project.id=81be097728134d76957fdb5755382cc2&role.id=86f185569d134d80841142ae8e9d3f47 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:00:00.705605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c3281-3441-426c-a4d8-966602ebba49 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:00.712142 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:00.715344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:00.758443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567db0d-d857-47c3-9495-d3d2813eccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:00.759493 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567db0d-d857-47c3-9495-d3d2813eccaa None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:00.759953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567db0d-d857-47c3-9495-d3d2813eccaa None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:00.760394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567db0d-d857-47c3-9495-d3d2813eccaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:00.919305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567db0d-d857-47c3-9495-d3d2813eccaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:00.921216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567db0d-d857-47c3-9495-d3d2813eccaa None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:00.997490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c3281-3441-426c-a4d8-966602ebba49 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:01.065883 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67299ecc-9b4e-44a9-a82b-40ef647fea10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:01.069820 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67299ecc-9b4e-44a9-a82b-40ef647fea10 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:01.070470 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67299ecc-9b4e-44a9-a82b-40ef647fea10 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:01.071131 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67299ecc-9b4e-44a9-a82b-40ef647fea10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:01.100628 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 180/357] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:01.103287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b458a9-38ba-4a00-81b9-a5c723bc6111 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:01.104099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b458a9-38ba-4a00-81b9-a5c723bc6111 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:01.104816 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b458a9-38ba-4a00-81b9-a5c723bc6111 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:01.108031 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 181/358] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:01.116877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67299ecc-9b4e-44a9-a82b-40ef647fea10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:01.117983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b458a9-38ba-4a00-81b9-a5c723bc6111 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-67299ecc-9b4e-44a9-a82b-40ef647fea10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:01.128780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:01.130641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:01.132242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:01.132242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:01.132242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:01.132242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:01.132242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:01.132242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:01.132242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:01.132242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:01.132242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:01.132242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:01.132242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:00:01.132242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:01.132242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 178/359] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:01.133545 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ff1f6-e597-41e2-bda7-ca0b8dae2ea7 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:01.134549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ff1f6-e597-41e2-bda7-ca0b8dae2ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:01.134549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ff1f6-e597-41e2-bda7-ca0b8dae2ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:01.168361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:01.181087 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-2f8ff1f6-e597-41e2-bda7-ca0b8dae2ea7 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:01.183784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2f8ff1f6-e597-41e2-bda7-ca0b8dae2ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:01.309497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:01.329734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 179/360] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:01.331830 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ff1f6-e597-41e2-bda7-ca0b8dae2ea7 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:01.374685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b458a9-38ba-4a00-81b9-a5c723bc6111 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:01.463207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5855e8-cac5-42af-9566-a6606edd063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:01.464201 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5855e8-cac5-42af-9566-a6606edd063c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:01.464201 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5855e8-cac5-42af-9566-a6606edd063c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:01.464490 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5855e8-cac5-42af-9566-a6606edd063c None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:01.465980 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5855e8-cac5-42af-9566-a6606edd063c None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:01.467543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5855e8-cac5-42af-9566-a6606edd063c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:01.469338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 182/361] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:01 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:01.470179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5855e8-cac5-42af-9566-a6606edd063c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:01.581126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ff1f6-e597-41e2-bda7-ca0b8dae2ea7 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:01.633085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:01.651308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1001652-7fff-4cc1-8d85-dd0434ce4cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:01.653943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1001652-7fff-4cc1-8d85-dd0434ce4cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:01.654803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1001652-7fff-4cc1-8d85-dd0434ce4cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:01.655600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1001652-7fff-4cc1-8d85-dd0434ce4cf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:01.658350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1001652-7fff-4cc1-8d85-dd0434ce4cf8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:01.659320 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1001652-7fff-4cc1-8d85-dd0434ce4cf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:01.662860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1001652-7fff-4cc1-8d85-dd0434ce4cf8 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:01.665062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5855e8-cac5-42af-9566-a6606edd063c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:01.681699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 180/362] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 13:00:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:01.691329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1001652-7fff-4cc1-8d85-dd0434ce4cf8 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:01.713012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04b591-95d3-4311-b8f7-f2228249b330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:01.713679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04b591-95d3-4311-b8f7-f2228249b330 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:01.713957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04b591-95d3-4311-b8f7-f2228249b330 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:01.714215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04b591-95d3-4311-b8f7-f2228249b330 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:01.729387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04b591-95d3-4311-b8f7-f2228249b330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-5f04b591-95d3-4311-b8f7-f2228249b330 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:01.735571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:01.737171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 183/363] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:00:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:01.738572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04b591-95d3-4311-b8f7-f2228249b330 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:01.801176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:01.889499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1001652-7fff-4cc1-8d85-dd0434ce4cf8 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:01.915904 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04b591-95d3-4311-b8f7-f2228249b330 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:01.960600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1d68f-2b28-4809-9c71-2ba3e8d47707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:01.961270 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1d68f-2b28-4809-9c71-2ba3e8d47707 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:01.961554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1d68f-2b28-4809-9c71-2ba3e8d47707 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:01.961835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1d68f-2b28-4809-9c71-2ba3e8d47707 None admin] PATH_INFO: `/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:01.963063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1d68f-2b28-4809-9c71-2ba3e8d47707 None admin] RBAC: Authorizing `identity:get_group(group_id=0be5c6a5a91743e3a1e5a7676f39026e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:01.964633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1d68f-2b28-4809-9c71-2ba3e8d47707 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:01.966456 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 181/364] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:01 2022] GET /identity/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:01.967231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1d68f-2b28-4809-9c71-2ba3e8d47707 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:01.980474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c067263-e48a-4ac8-937f-990c00e95f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:01.981916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c067263-e48a-4ac8-937f-990c00e95f3d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:01.982343 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c067263-e48a-4ac8-937f-990c00e95f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:01.982745 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c067263-e48a-4ac8-937f-990c00e95f3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:02.009399 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:02.096968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c067263-e48a-4ac8-937f-990c00e95f3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:02.098541 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c067263-e48a-4ac8-937f-990c00e95f3d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:02.149483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1d68f-2b28-4809-9c71-2ba3e8d47707 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:02.207510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:02.232936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b8df5-e2c6-4370-8c54-cfd69e06cd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:02.233590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b8df5-e2c6-4370-8c54-cfd69e06cd8d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:02.233870 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b8df5-e2c6-4370-8c54-cfd69e06cd8d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:02.234140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b8df5-e2c6-4370-8c54-cfd69e06cd8d None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:02.234943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b8df5-e2c6-4370-8c54-cfd69e06cd8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:02.235329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b8df5-e2c6-4370-8c54-cfd69e06cd8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:02.236944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b8df5-e2c6-4370-8c54-cfd69e06cd8d None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:02.253108 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 182/365] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:02.254209 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b8df5-e2c6-4370-8c54-cfd69e06cd8d None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:02.301076 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 184/366] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:02.303006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c067263-e48a-4ac8-937f-990c00e95f3d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:02.406954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:02.443011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b8df5-e2c6-4370-8c54-cfd69e06cd8d None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:02.519053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d837e-cbe6-443c-b468-f7757a246b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:02.520648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d837e-cbe6-443c-b468-f7757a246b5b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:02.521383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d837e-cbe6-443c-b468-f7757a246b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:02.522115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d837e-cbe6-443c-b468-f7757a246b5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:02.527448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c067263-e48a-4ac8-937f-990c00e95f3d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:02.547218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d837e-cbe6-443c-b468-f7757a246b5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-a96d837e-cbe6-443c-b468-f7757a246b5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:02.559921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:02.561644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 183/367] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:00:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:02.564281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d837e-cbe6-443c-b468-f7757a246b5b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:02.588249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dc289-fd4e-4f69-9bee-f76b972d36d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:02.588906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dc289-fd4e-4f69-9bee-f76b972d36d9 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:02.589158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dc289-fd4e-4f69-9bee-f76b972d36d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:02.589976 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dc289-fd4e-4f69-9bee-f76b972d36d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:02.590748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dc289-fd4e-4f69-9bee-f76b972d36d9 None admin] RBAC: Adding query filter params (group.id=0be5c6a5a91743e3a1e5a7676f39026e, role.id=649721c72e5d40deabd3512c529ffdf4, scope.project.id=81be097728134d76957fdb5755382cc2) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:02.591059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dc289-fd4e-4f69-9bee-f76b972d36d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:02.593664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dc289-fd4e-4f69-9bee-f76b972d36d9 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:02.602957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 185/368] 10.222.0.61 () {56 vars in 1376 bytes} [Wed Sep 7 13:00:02 2022] GET /identity/v3/role_assignments?group.id=0be5c6a5a91743e3a1e5a7676f39026e&scope.project.id=81be097728134d76957fdb5755382cc2&role.id=649721c72e5d40deabd3512c529ffdf4 => generated 260 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:02.604138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dc289-fd4e-4f69-9bee-f76b972d36d9 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:02.648417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:02.758633 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dc289-fd4e-4f69-9bee-f76b972d36d9 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:02.766841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:02.775135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d837e-cbe6-443c-b468-f7757a246b5b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:02.812606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54ea0c-df77-4349-b98f-9fb32f0b7c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:02.813514 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54ea0c-df77-4349-b98f-9fb32f0b7c2d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:02.813804 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54ea0c-df77-4349-b98f-9fb32f0b7c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:02.814075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54ea0c-df77-4349-b98f-9fb32f0b7c2d None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:02.815706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca54ea0c-df77-4349-b98f-9fb32f0b7c2d None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:02.817946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca54ea0c-df77-4349-b98f-9fb32f0b7c2d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:02.820009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 186/369] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:02 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:02.820918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54ea0c-df77-4349-b98f-9fb32f0b7c2d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:02.857831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048a9e4-cd67-46ae-92a9-677f0cdc2dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:02.859594 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048a9e4-cd67-46ae-92a9-677f0cdc2dbb None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:02.860621 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048a9e4-cd67-46ae-92a9-677f0cdc2dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:02.861464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048a9e4-cd67-46ae-92a9-677f0cdc2dbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:02.990909 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048a9e4-cd67-46ae-92a9-677f0cdc2dbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:02.992460 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048a9e4-cd67-46ae-92a9-677f0cdc2dbb None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:02.993774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54ea0c-df77-4349-b98f-9fb32f0b7c2d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:03.056803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:03.086845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70139fc-bb43-4171-bc57-5107d81c9748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:03.088134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70139fc-bb43-4171-bc57-5107d81c9748 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:03.088683 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70139fc-bb43-4171-bc57-5107d81c9748 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:03.089388 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70139fc-bb43-4171-bc57-5107d81c9748 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:03.090721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70139fc-bb43-4171-bc57-5107d81c9748 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:03.091666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70139fc-bb43-4171-bc57-5107d81c9748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:03.094207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70139fc-bb43-4171-bc57-5107d81c9748 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:03.114530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 187/370] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:03.119890 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70139fc-bb43-4171-bc57-5107d81c9748 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:03.138628 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 184/371] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:03.140520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048a9e4-cd67-46ae-92a9-677f0cdc2dbb None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:03.197502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:03.290751 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048a9e4-cd67-46ae-92a9-677f0cdc2dbb None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:03.342430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70139fc-bb43-4171-bc57-5107d81c9748 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:03.358229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c55ba0-a8cb-4936-83ef-73e71ea2089c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:03.358881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c55ba0-a8cb-4936-83ef-73e71ea2089c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:03.359198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c55ba0-a8cb-4936-83ef-73e71ea2089c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:03.359482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c55ba0-a8cb-4936-83ef-73e71ea2089c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:03.389510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:03.400864 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e6aaa-563a-4ee0-a2c8-93cc5906ff32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:03.401561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e6aaa-563a-4ee0-a2c8-93cc5906ff32 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:03.402498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e6aaa-563a-4ee0-a2c8-93cc5906ff32 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:03.402498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e6aaa-563a-4ee0-a2c8-93cc5906ff32 None admin] PATH_INFO: `/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:03.403479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e6aaa-563a-4ee0-a2c8-93cc5906ff32 None admin] RBAC: Authorizing `identity:get_group(group_id=0be5c6a5a91743e3a1e5a7676f39026e)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:03.404998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e6aaa-563a-4ee0-a2c8-93cc5906ff32 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:03.407481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 188/372] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:03 2022] GET /identity/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:03.409230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e6aaa-563a-4ee0-a2c8-93cc5906ff32 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:03.488083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c55ba0-a8cb-4936-83ef-73e71ea2089c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:03.489787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c55ba0-a8cb-4936-83ef-73e71ea2089c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:03.597142 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e6aaa-563a-4ee0-a2c8-93cc5906ff32 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:03.602901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:03.635890 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 185/373] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:03.637933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c55ba0-a8cb-4936-83ef-73e71ea2089c None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:03.668044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fb02f-762a-47cb-8087-98e72b03ad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:03.668715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fb02f-762a-47cb-8087-98e72b03ad29 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:03.668992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fb02f-762a-47cb-8087-98e72b03ad29 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:03.669262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fb02f-762a-47cb-8087-98e72b03ad29 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:03.670547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fb02f-762a-47cb-8087-98e72b03ad29 None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:03.672102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fb02f-762a-47cb-8087-98e72b03ad29 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:03.674042 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 189/374] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:03 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:03.674867 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fb02f-762a-47cb-8087-98e72b03ad29 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:03.721376 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:03.792055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c55ba0-a8cb-4936-83ef-73e71ea2089c None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:03.813989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fb02f-762a-47cb-8087-98e72b03ad29 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:03.841530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44bf9-8299-4ba7-9145-f0093b266764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:03.842339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44bf9-8299-4ba7-9145-f0093b266764 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:03.842789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44bf9-8299-4ba7-9145-f0093b266764 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:03.843152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f44bf9-8299-4ba7-9145-f0093b266764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:03.844042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f44bf9-8299-4ba7-9145-f0093b266764 None admin] RBAC: Adding query filter params (group.id=0be5c6a5a91743e3a1e5a7676f39026e, role.id=86f185569d134d80841142ae8e9d3f47, scope.project.id=717d2ed44c684799a826e03b360f281b) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:03.844490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f44bf9-8299-4ba7-9145-f0093b266764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:03.846761 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f44bf9-8299-4ba7-9145-f0093b266764 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:03.853711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 186/375] 10.222.0.61 () {56 vars in 1376 bytes} [Wed Sep 7 13:00:03 2022] GET /identity/v3/role_assignments?group.id=0be5c6a5a91743e3a1e5a7676f39026e&scope.project.id=717d2ed44c684799a826e03b360f281b&role.id=86f185569d134d80841142ae8e9d3f47 => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:03.854478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44bf9-8299-4ba7-9145-f0093b266764 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:03.861538 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b88ac-5550-4200-9020-57629db020d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:03.862597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b88ac-5550-4200-9020-57629db020d5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:03.863031 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b88ac-5550-4200-9020-57629db020d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:03.863602 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b88ac-5550-4200-9020-57629db020d5 None admin] PATH_INFO: `/v3/projects/2f256a069ba947c6a6b8cccfea6c7a22` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:03.865251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b88ac-5550-4200-9020-57629db020d5 None admin] RBAC: Authorizing `identity:get_project(project_id=2f256a069ba947c6a6b8cccfea6c7a22)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:03.867288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b88ac-5550-4200-9020-57629db020d5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:03.870220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 190/376] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:03 2022] GET /identity/v3/projects/2f256a069ba947c6a6b8cccfea6c7a22 => generated 344 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:03.884827 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b88ac-5550-4200-9020-57629db020d5 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:03.959817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:04.008378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f44bf9-8299-4ba7-9145-f0093b266764 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:04.026227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:04.059779 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69854bb4-6757-49ce-a544-62e604e73af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:04.060558 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69854bb4-6757-49ce-a544-62e604e73af6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:04.060925 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69854bb4-6757-49ce-a544-62e604e73af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:04.061275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69854bb4-6757-49ce-a544-62e604e73af6 None admin] PATH_INFO: `/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:04.062657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69854bb4-6757-49ce-a544-62e604e73af6 None admin] RBAC: Authorizing `identity:get_group(group_id=0be5c6a5a91743e3a1e5a7676f39026e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:04.064324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69854bb4-6757-49ce-a544-62e604e73af6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:04.066640 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 187/377] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:03 2022] GET /identity/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:04.076271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69854bb4-6757-49ce-a544-62e604e73af6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:04.126179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b88ac-5550-4200-9020-57629db020d5 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:04.205874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0e6ea-925c-436b-954b-0fea77a090e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:04.207445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0e6ea-925c-436b-954b-0fea77a090e6 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:04.208149 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0e6ea-925c-436b-954b-0fea77a090e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:04.208901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0e6ea-925c-436b-954b-0fea77a090e6 None admin] PATH_INFO: `/v3/groups/f3370f3537e04f36bca6f3d3b95c79b0` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:04.222869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0e6ea-925c-436b-954b-0fea77a090e6 None admin] RBAC: Authorizing `identity:get_group(group_id=f3370f3537e04f36bca6f3d3b95c79b0)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:04.226212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0e6ea-925c-436b-954b-0fea77a090e6 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:04.231126 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 191/378] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:03 2022] GET /identity/v3/groups/f3370f3537e04f36bca6f3d3b95c79b0 => generated 229 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:04.240953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0e6ea-925c-436b-954b-0fea77a090e6 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:04.267786 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69854bb4-6757-49ce-a544-62e604e73af6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:04.370129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9fd523-9758-46de-a984-b2908259cb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:04.372301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fd523-9758-46de-a984-b2908259cb38 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:04.376450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fd523-9758-46de-a984-b2908259cb38 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:04.377302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fd523-9758-46de-a984-b2908259cb38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:04.379863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9fd523-9758-46de-a984-b2908259cb38 None admin] RBAC: Adding query filter params (group.id=0be5c6a5a91743e3a1e5a7676f39026e, role.id=649721c72e5d40deabd3512c529ffdf4, scope.project.id=717d2ed44c684799a826e03b360f281b) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:04.380203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9fd523-9758-46de-a984-b2908259cb38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:04.381883 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9fd523-9758-46de-a984-b2908259cb38 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:04.389370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 188/379] 10.222.0.61 () {56 vars in 1376 bytes} [Wed Sep 7 13:00:04 2022] GET /identity/v3/role_assignments?group.id=0be5c6a5a91743e3a1e5a7676f39026e&scope.project.id=717d2ed44c684799a826e03b360f281b&role.id=649721c72e5d40deabd3512c529ffdf4 => generated 260 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:04.483910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0e6ea-925c-436b-954b-0fea77a090e6 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:04.489150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:04.549726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be347a2-8c8d-4aed-903f-0a8c85b95c3c None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:04.550515 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be347a2-8c8d-4aed-903f-0a8c85b95c3c None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:04.550857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be347a2-8c8d-4aed-903f-0a8c85b95c3c None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:04.555897 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 189/380] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:04.565322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5657c8-bc01-4c70-814c-b9c1764a214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:04.566879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5657c8-bc01-4c70-814c-b9c1764a214d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:04.567038 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955898fa-9538-4d67-af5e-319edb874dfb None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:04.567509 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955898fa-9538-4d67-af5e-319edb874dfb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:04.567999 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955898fa-9538-4d67-af5e-319edb874dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:04.568409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5657c8-bc01-4c70-814c-b9c1764a214d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:04.569050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5657c8-bc01-4c70-814c-b9c1764a214d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:04.570510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5657c8-bc01-4c70-814c-b9c1764a214d None admin] RBAC: Adding query filter params (group.id=f3370f3537e04f36bca6f3d3b95c79b0, role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=2f256a069ba947c6a6b8cccfea6c7a22) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:04.571296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5657c8-bc01-4c70-814c-b9c1764a214d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:04.574153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5657c8-bc01-4c70-814c-b9c1764a214d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:04.587314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 192/381] 10.222.0.61 () {56 vars in 1376 bytes} [Wed Sep 7 13:00:04 2022] GET /identity/v3/role_assignments?group.id=f3370f3537e04f36bca6f3d3b95c79b0&scope.project.id=2f256a069ba947c6a6b8cccfea6c7a22&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 260 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:04.615776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-955898fa-9538-4d67-af5e-319edb874dfb None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:04.618584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-955898fa-9538-4d67-af5e-319edb874dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:04.709201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:04.769850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 190/382] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:00:05.413666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5657c8-bc01-4c70-814c-b9c1764a214d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:05.606083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5657c8-bc01-4c70-814c-b9c1764a214d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:05.661315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932370b-cd11-4f79-890a-a66deab33917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:05.661995 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932370b-cd11-4f79-890a-a66deab33917 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:05.662271 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932370b-cd11-4f79-890a-a66deab33917 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:05.662536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932370b-cd11-4f79-890a-a66deab33917 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:05.677924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932370b-cd11-4f79-890a-a66deab33917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-9932370b-cd11-4f79-890a-a66deab33917 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:05.684229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:05.685198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:05.686011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:05.686011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:05.686011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:05.686011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:05.686011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:05.686011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:05.686011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:05.686011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:05.686011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:05.686011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:05.686011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:00:05.686011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:05.686011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 193/383] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:05.694607 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955898fa-9538-4d67-af5e-319edb874dfb None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:05.843341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955898fa-9538-4d67-af5e-319edb874dfb None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:05.865195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a7b1f-a4d5-4c1a-b50a-5970cde4bd15 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:05.865700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a7b1f-a4d5-4c1a-b50a-5970cde4bd15 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:05.866187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a7b1f-a4d5-4c1a-b50a-5970cde4bd15 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:05.879295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 194/384] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:00:05.886189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8847ff-6deb-4ca3-8d79-3386f9ce3a13 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:05.886494 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8847ff-6deb-4ca3-8d79-3386f9ce3a13 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:05.886746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8847ff-6deb-4ca3-8d79-3386f9ce3a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:05.918056 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55838d73-3395-4d70-bd99-d8f0cfed690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:05.919162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55838d73-3395-4d70-bd99-d8f0cfed690e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:05.919665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55838d73-3395-4d70-bd99-d8f0cfed690e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:05.920170 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55838d73-3395-4d70-bd99-d8f0cfed690e None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:05.921428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55838d73-3395-4d70-bd99-d8f0cfed690e None admin] RBAC: Adding query filter params (name=member) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:05.921974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55838d73-3395-4d70-bd99-d8f0cfed690e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:05.924648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55838d73-3395-4d70-bd99-d8f0cfed690e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:05.950059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 191/385] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 13:00:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:05.951899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-0c8847ff-6deb-4ca3-8d79-3386f9ce3a13 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:05.954115 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0c8847ff-6deb-4ca3-8d79-3386f9ce3a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:05.980281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55838d73-3395-4d70-bd99-d8f0cfed690e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:06.014651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:06.148112 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:06.169487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 195/386] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:06.290646 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55838d73-3395-4d70-bd99-d8f0cfed690e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:06.374633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511636a5-5479-4307-960e-5860c04a9628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:06.375757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511636a5-5479-4307-960e-5860c04a9628 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:06.376239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511636a5-5479-4307-960e-5860c04a9628 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:06.376728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511636a5-5479-4307-960e-5860c04a9628 None admin] PATH_INFO: `/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:06.379963 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511636a5-5479-4307-960e-5860c04a9628 None admin] RBAC: Authorizing `identity:get_group(group_id=0be5c6a5a91743e3a1e5a7676f39026e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:06.381498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511636a5-5479-4307-960e-5860c04a9628 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:06.383642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 192/387] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:05 2022] GET /identity/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e => generated 226 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:06.393089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8847ff-6deb-4ca3-8d79-3386f9ce3a13 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:06.562003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8847ff-6deb-4ca3-8d79-3386f9ce3a13 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:06.629103 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb35fed-9437-4cc1-9cf4-9d312c08d117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:06.629774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb35fed-9437-4cc1-9cf4-9d312c08d117 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:06.630108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb35fed-9437-4cc1-9cf4-9d312c08d117 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:06.630384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb35fed-9437-4cc1-9cf4-9d312c08d117 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:06.631740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb35fed-9437-4cc1-9cf4-9d312c08d117 None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:06.633507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb35fed-9437-4cc1-9cf4-9d312c08d117 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:06.635608 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 196/388] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:06 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:00:06.645374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511636a5-5479-4307-960e-5860c04a9628 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:06.822780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511636a5-5479-4307-960e-5860c04a9628 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:06.824789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:06.838083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb35fed-9437-4cc1-9cf4-9d312c08d117 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:06.896352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780f516-404b-47b7-85d5-ae8d3b825855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:06.897057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780f516-404b-47b7-85d5-ae8d3b825855 None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:06.897336 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780f516-404b-47b7-85d5-ae8d3b825855 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:06.897611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780f516-404b-47b7-85d5-ae8d3b825855 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2/groups/0be5c6a5a91743e3a1e5a7676f39026e/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:06.902489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780f516-404b-47b7-85d5-ae8d3b825855 None admin] RBAC: Authorizing `identity:create_grant(project_id=81be097728134d76957fdb5755382cc2, group_id=0be5c6a5a91743e3a1e5a7676f39026e, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:06.904044 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780f516-404b-47b7-85d5-ae8d3b825855 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:06.911231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:06.932874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 193/389] 10.222.0.61 () {58 vars in 1442 bytes} [Wed Sep 7 13:00:06 2022] PUT /identity/v3/projects/81be097728134d76957fdb5755382cc2/groups/0be5c6a5a91743e3a1e5a7676f39026e/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:00:07.029538 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb35fed-9437-4cc1-9cf4-9d312c08d117 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:07.079251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116bf60-545d-4a67-a0b4-a9271026e656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:07.079251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116bf60-545d-4a67-a0b4-a9271026e656 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:07.079251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116bf60-545d-4a67-a0b4-a9271026e656 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:07.079251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116bf60-545d-4a67-a0b4-a9271026e656 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116bf60-545d-4a67-a0b4-a9271026e656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-0116bf60-545d-4a67-a0b4-a9271026e656 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:07.099172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:07.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:07.104260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 197/390] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:00:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:07.111867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780f516-404b-47b7-85d5-ae8d3b825855 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:07.259754 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780f516-404b-47b7-85d5-ae8d3b825855 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:07.303632 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f220e-061d-4119-a758-aebb53f73c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:07.304277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f220e-061d-4119-a758-aebb53f73c84 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:07.304546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f220e-061d-4119-a758-aebb53f73c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:07.304795 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f220e-061d-4119-a758-aebb53f73c84 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:07.305523 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f220e-061d-4119-a758-aebb53f73c84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:07.305835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f220e-061d-4119-a758-aebb53f73c84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:07.306911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55758b6-dd7b-4e81-987f-5ddf5b017939 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:07.310524 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f220e-061d-4119-a758-aebb53f73c84 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:07.312513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55758b6-dd7b-4e81-987f-5ddf5b017939 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:07.313407 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55758b6-dd7b-4e81-987f-5ddf5b017939 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:07.318221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 194/391] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:07.320995 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 198/392] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:07 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:00:07.324166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f220e-061d-4119-a758-aebb53f73c84 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:07.325365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbc0dc-9bb2-4c3b-8635-5d39d864f00c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:07.325654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbc0dc-9bb2-4c3b-8635-5d39d864f00c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:07.325929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbc0dc-9bb2-4c3b-8635-5d39d864f00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:07.364910 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:07.369489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-10bbc0dc-9bb2-4c3b-8635-5d39d864f00c None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:07.372518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-10bbc0dc-9bb2-4c3b-8635-5d39d864f00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:07.467815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f220e-061d-4119-a758-aebb53f73c84 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:07.508302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a7c67-8f6c-4b1c-b2b5-838131e68bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:07.509309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a7c67-8f6c-4b1c-b2b5-838131e68bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:07.509775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a7c67-8f6c-4b1c-b2b5-838131e68bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:07.510238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a7c67-8f6c-4b1c-b2b5-838131e68bc7 None admin] PATH_INFO: `/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:07.511802 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a7c67-8f6c-4b1c-b2b5-838131e68bc7 None admin] RBAC: Authorizing `identity:get_group(group_id=0be5c6a5a91743e3a1e5a7676f39026e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:07.513606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a7c67-8f6c-4b1c-b2b5-838131e68bc7 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:07.516059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 195/393] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:07 2022] GET /identity/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:07.522143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a7c67-8f6c-4b1c-b2b5-838131e68bc7 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:07.537111 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 199/394] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:07.648167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a7c67-8f6c-4b1c-b2b5-838131e68bc7 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:07.652444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:07.692365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5436756-92bc-4b60-9c5b-2a1fb9405809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:07.693402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5436756-92bc-4b60-9c5b-2a1fb9405809 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:07.693901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5436756-92bc-4b60-9c5b-2a1fb9405809 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:07.694365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5436756-92bc-4b60-9c5b-2a1fb9405809 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:07.696106 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5436756-92bc-4b60-9c5b-2a1fb9405809 None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:07.697880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5436756-92bc-4b60-9c5b-2a1fb9405809 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:07.700609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 196/395] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:07 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:00:07.707614 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbc0dc-9bb2-4c3b-8635-5d39d864f00c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:07.736303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:07.890952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027a9f9-3a87-4a3e-aaf3-9be8ba50f1cb None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:07.892067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027a9f9-3a87-4a3e-aaf3-9be8ba50f1cb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:07.892405 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027a9f9-3a87-4a3e-aaf3-9be8ba50f1cb None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:07.894265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 197/396] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:07.903457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb864c40-362f-41b3-b7c7-2773ceb582f5 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:07.903692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb864c40-362f-41b3-b7c7-2773ceb582f5 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:07.903956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb864c40-362f-41b3-b7c7-2773ceb582f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:07.932834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbc0dc-9bb2-4c3b-8635-5d39d864f00c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:07.967348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-fb864c40-362f-41b3-b7c7-2773ceb582f5 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:07.969426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-fb864c40-362f-41b3-b7c7-2773ceb582f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:07.974987 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfec6b1-02fd-4db0-9dda-e024cfcf89a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:07.975842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfec6b1-02fd-4db0-9dda-e024cfcf89a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:07.976128 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfec6b1-02fd-4db0-9dda-e024cfcf89a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:07.976410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfec6b1-02fd-4db0-9dda-e024cfcf89a7 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2/groups/0be5c6a5a91743e3a1e5a7676f39026e/roles/649721c72e5d40deabd3512c529ffdf4` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:07.979130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfec6b1-02fd-4db0-9dda-e024cfcf89a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=81be097728134d76957fdb5755382cc2, group_id=0be5c6a5a91743e3a1e5a7676f39026e, role_id=649721c72e5d40deabd3512c529ffdf4)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:07.981428 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfec6b1-02fd-4db0-9dda-e024cfcf89a7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:07.998336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 200/397] 10.222.0.61 () {58 vars in 1442 bytes} [Wed Sep 7 13:00:07 2022] PUT /identity/v3/projects/81be097728134d76957fdb5755382cc2/groups/0be5c6a5a91743e3a1e5a7676f39026e/roles/649721c72e5d40deabd3512c529ffdf4 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:00:07.999051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfec6b1-02fd-4db0-9dda-e024cfcf89a7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:08.101129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:08.119447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 198/398] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:08.133854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfec6b1-02fd-4db0-9dda-e024cfcf89a7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:08.181228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d387ff0-dd44-49e6-a62f-9fd6f0f9e9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:08.181825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d387ff0-dd44-49e6-a62f-9fd6f0f9e9ac None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:08.182101 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d387ff0-dd44-49e6-a62f-9fd6f0f9e9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:08.182384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d387ff0-dd44-49e6-a62f-9fd6f0f9e9ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:08.196606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d387ff0-dd44-49e6-a62f-9fd6f0f9e9ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-8d387ff0-dd44-49e6-a62f-9fd6f0f9e9ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:08.203881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:08.206333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:08.209563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:08.209563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:08.209563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:08.209563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:08.209563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:08.209563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:08.209563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:08.209563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:08.209563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:08.209563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:08.209563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:00:08.209563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:08.209563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 201/399] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:08.225015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb864c40-362f-41b3-b7c7-2773ceb582f5 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:08.433955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:08.453584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb864c40-362f-41b3-b7c7-2773ceb582f5 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:08.522639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1037e2-3a28-4553-8956-2620dd0f57de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:08.523381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1037e2-3a28-4553-8956-2620dd0f57de None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:08.523858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1037e2-3a28-4553-8956-2620dd0f57de None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:08.524415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1037e2-3a28-4553-8956-2620dd0f57de None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:08.525652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1037e2-3a28-4553-8956-2620dd0f57de None admin] RBAC: Adding query filter params (name=member) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:08.526190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1037e2-3a28-4553-8956-2620dd0f57de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:08.531983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1037e2-3a28-4553-8956-2620dd0f57de None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:08.548160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 199/400] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 13:00:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:08.557139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d387ff0-dd44-49e6-a62f-9fd6f0f9e9ac None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:08.688718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1037e2-3a28-4553-8956-2620dd0f57de None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:08.757566 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d387ff0-dd44-49e6-a62f-9fd6f0f9e9ac None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:08.805289 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c5f92-a739-47c2-9c2d-93c6e1591990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:08.805956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c5f92-a739-47c2-9c2d-93c6e1591990 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:08.806244 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c5f92-a739-47c2-9c2d-93c6e1591990 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:08.806519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c5f92-a739-47c2-9c2d-93c6e1591990 None admin] PATH_INFO: `/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:08.807874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c5f92-a739-47c2-9c2d-93c6e1591990 None admin] RBAC: Authorizing `identity:get_group(group_id=0be5c6a5a91743e3a1e5a7676f39026e)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:08.809923 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c5f92-a739-47c2-9c2d-93c6e1591990 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:08.812091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 202/401] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:08 2022] GET /identity/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:08.813431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f295b13-be74-4d97-84bf-d2d591e76bae None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:08.813734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f295b13-be74-4d97-84bf-d2d591e76bae None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:08.814001 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f295b13-be74-4d97-84bf-d2d591e76bae None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:08.815775 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 203/402] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:08.820957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f295b13-be74-4d97-84bf-d2d591e76bae None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:08.825187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:08.855849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1037e2-3a28-4553-8956-2620dd0f57de None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:08.901366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33ef22-aefd-47f3-9944-d805e345aff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:08.902138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33ef22-aefd-47f3-9944-d805e345aff4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:08.902430 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33ef22-aefd-47f3-9944-d805e345aff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:08.902764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33ef22-aefd-47f3-9944-d805e345aff4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:08.916586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33ef22-aefd-47f3-9944-d805e345aff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-8a33ef22-aefd-47f3-9944-d805e345aff4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:08.928829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:08.929956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:08.930845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:08.930845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:08.930845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:08.930845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:08.930845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:08.930845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:08.930845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:08.930845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:08.930845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:08.930845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:08.930845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:00:08.930845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:08.935326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 200/403] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:00:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:08.937719 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f27d91-5253-4217-b361-795d47200d76 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:08.938222 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f27d91-5253-4217-b361-795d47200d76 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:08.938685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f27d91-5253-4217-b361-795d47200d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:08.952086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f295b13-be74-4d97-84bf-d2d591e76bae None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:08.961388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-36f27d91-5253-4217-b361-795d47200d76 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:08.962689 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-36f27d91-5253-4217-b361-795d47200d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:08.975275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:08.994498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718406e-85a9-46af-9ca9-846ec7e02b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:08.996635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718406e-85a9-46af-9ca9-846ec7e02b13 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:08.997108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718406e-85a9-46af-9ca9-846ec7e02b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:08.997812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718406e-85a9-46af-9ca9-846ec7e02b13 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:08.999508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718406e-85a9-46af-9ca9-846ec7e02b13 None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:09.001908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718406e-85a9-46af-9ca9-846ec7e02b13 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:09.005385 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 204/404] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:08 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:09.006512 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718406e-85a9-46af-9ca9-846ec7e02b13 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:09.067628 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 201/405] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:09.069274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f27d91-5253-4217-b361-795d47200d76 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:09.099822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:09.139606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718406e-85a9-46af-9ca9-846ec7e02b13 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:09.183884 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73161b3b-22c3-4d12-a1bb-3fba81ddcecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:09.184526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73161b3b-22c3-4d12-a1bb-3fba81ddcecd None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:09.184810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73161b3b-22c3-4d12-a1bb-3fba81ddcecd None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:09.185089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73161b3b-22c3-4d12-a1bb-3fba81ddcecd None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:09.186009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73161b3b-22c3-4d12-a1bb-3fba81ddcecd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:09.186446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73161b3b-22c3-4d12-a1bb-3fba81ddcecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:09.187920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73161b3b-22c3-4d12-a1bb-3fba81ddcecd None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:09.199664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 205/406] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:09.210576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73161b3b-22c3-4d12-a1bb-3fba81ddcecd None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:09.318740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f27d91-5253-4217-b361-795d47200d76 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:09.389718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a31f5a-d5e2-4268-8378-5cf3861129d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:09.391049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a31f5a-d5e2-4268-8378-5cf3861129d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:09.391629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a31f5a-d5e2-4268-8378-5cf3861129d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:09.392173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a31f5a-d5e2-4268-8378-5cf3861129d4 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b/groups/0be5c6a5a91743e3a1e5a7676f39026e/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:09.394742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a31f5a-d5e2-4268-8378-5cf3861129d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=717d2ed44c684799a826e03b360f281b, group_id=0be5c6a5a91743e3a1e5a7676f39026e, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:09.396575 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a31f5a-d5e2-4268-8378-5cf3861129d4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:09.421663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 202/407] 10.222.0.61 () {58 vars in 1442 bytes} [Wed Sep 7 13:00:09 2022] PUT /identity/v3/projects/717d2ed44c684799a826e03b360f281b/groups/0be5c6a5a91743e3a1e5a7676f39026e/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:00:09.442862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73161b3b-22c3-4d12-a1bb-3fba81ddcecd None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:09.457088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a31f5a-d5e2-4268-8378-5cf3861129d4 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:09.468457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:09.490836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074dc3a-0bbd-4a20-8646-12ae84abf68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:09.491959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074dc3a-0bbd-4a20-8646-12ae84abf68f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:09.492477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074dc3a-0bbd-4a20-8646-12ae84abf68f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:09.492996 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074dc3a-0bbd-4a20-8646-12ae84abf68f None admin] PATH_INFO: `/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:09.494885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074dc3a-0bbd-4a20-8646-12ae84abf68f None admin] RBAC: Authorizing `identity:get_group(group_id=0be5c6a5a91743e3a1e5a7676f39026e)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:09.496775 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074dc3a-0bbd-4a20-8646-12ae84abf68f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:09.499187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 206/408] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:09 2022] GET /identity/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:09.506151 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074dc3a-0bbd-4a20-8646-12ae84abf68f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:09.511687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:09.675571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074dc3a-0bbd-4a20-8646-12ae84abf68f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:09.720748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a31f5a-d5e2-4268-8378-5cf3861129d4 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:09.731265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cbb83-8ad8-4381-82fb-04bddb6fda94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:09.732719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cbb83-8ad8-4381-82fb-04bddb6fda94 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:09.733435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cbb83-8ad8-4381-82fb-04bddb6fda94 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:09.734049 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cbb83-8ad8-4381-82fb-04bddb6fda94 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:09.742811 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cbb83-8ad8-4381-82fb-04bddb6fda94 None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:09.742811 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cbb83-8ad8-4381-82fb-04bddb6fda94 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:09.743295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 207/409] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:09 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:09.754658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cbb83-8ad8-4381-82fb-04bddb6fda94 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:09.774084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ae80f-5e46-4d31-afac-34ea3a1accf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:09.774757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ae80f-5e46-4d31-afac-34ea3a1accf1 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:09.775038 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ae80f-5e46-4d31-afac-34ea3a1accf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:09.775353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ae80f-5e46-4d31-afac-34ea3a1accf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:09.788812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ae80f-5e46-4d31-afac-34ea3a1accf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-057ae80f-5e46-4d31-afac-34ea3a1accf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:09.794588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:09.795768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:09.796557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:09.796557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:09.796557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:09.796557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:09.796557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:09.796557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:09.796557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:09.796557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:09.796557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:09.796557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:09.796557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:00:09.796557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:09.796557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 203/410] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:00:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:09.802342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ae80f-5e46-4d31-afac-34ea3a1accf1 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:09.885257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:09.903695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:09.923552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cbb83-8ad8-4381-82fb-04bddb6fda94 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:09.929280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ae80f-5e46-4d31-afac-34ea3a1accf1 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:09.968876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87954ea2-a432-44e0-9e8c-1dd506c89b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:09.969228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b7078-18b7-4ff0-b87b-cb8906cca511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:09.969842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b7078-18b7-4ff0-b87b-cb8906cca511 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:09.970126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b7078-18b7-4ff0-b87b-cb8906cca511 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:09.970401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b7078-18b7-4ff0-b87b-cb8906cca511 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:09.970894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87954ea2-a432-44e0-9e8c-1dd506c89b13 None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:09.971247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b7078-18b7-4ff0-b87b-cb8906cca511 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:09.971663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87954ea2-a432-44e0-9e8c-1dd506c89b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:09.971792 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b7078-18b7-4ff0-b87b-cb8906cca511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:09.972197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87954ea2-a432-44e0-9e8c-1dd506c89b13 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b/groups/0be5c6a5a91743e3a1e5a7676f39026e/roles/649721c72e5d40deabd3512c529ffdf4` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:09.973380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b7078-18b7-4ff0-b87b-cb8906cca511 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:09.975877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87954ea2-a432-44e0-9e8c-1dd506c89b13 None admin] RBAC: Authorizing `identity:create_grant(project_id=717d2ed44c684799a826e03b360f281b, group_id=0be5c6a5a91743e3a1e5a7676f39026e, role_id=649721c72e5d40deabd3512c529ffdf4)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:09.978180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87954ea2-a432-44e0-9e8c-1dd506c89b13 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:09.984721 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 204/411] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:09.993195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b7078-18b7-4ff0-b87b-cb8906cca511 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:10.010416 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 208/412] 10.222.0.61 () {58 vars in 1442 bytes} [Wed Sep 7 13:00:09 2022] PUT /identity/v3/projects/717d2ed44c684799a826e03b360f281b/groups/0be5c6a5a91743e3a1e5a7676f39026e/roles/649721c72e5d40deabd3512c529ffdf4 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:00:10.135424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b7078-18b7-4ff0-b87b-cb8906cca511 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:10.169725 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:10.183637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0a5ba-0ba3-47e8-a1dc-6681197c37f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:10.184356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a5ba-0ba3-47e8-a1dc-6681197c37f5 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:10.184639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a5ba-0ba3-47e8-a1dc-6681197c37f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:10.184920 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a5ba-0ba3-47e8-a1dc-6681197c37f5 None admin] PATH_INFO: `/v3/groups/f3370f3537e04f36bca6f3d3b95c79b0` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:10.186154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0a5ba-0ba3-47e8-a1dc-6681197c37f5 None admin] RBAC: Authorizing `identity:get_group(group_id=f3370f3537e04f36bca6f3d3b95c79b0)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:10.187670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0a5ba-0ba3-47e8-a1dc-6681197c37f5 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:10.189647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 205/413] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:09 2022] GET /identity/v3/groups/f3370f3537e04f36bca6f3d3b95c79b0 => generated 229 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:10.196585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87954ea2-a432-44e0-9e8c-1dd506c89b13 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:10.357562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87954ea2-a432-44e0-9e8c-1dd506c89b13 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:10.415747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:10.428952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc062c9-bc78-4fa3-8fdb-96643be53db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:10.430134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc062c9-bc78-4fa3-8fdb-96643be53db9 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:10.431412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc062c9-bc78-4fa3-8fdb-96643be53db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:10.432730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc062c9-bc78-4fa3-8fdb-96643be53db9 None admin] PATH_INFO: `/v3/projects/2f256a069ba947c6a6b8cccfea6c7a22` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:10.436582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc062c9-bc78-4fa3-8fdb-96643be53db9 None admin] RBAC: Authorizing `identity:get_project(project_id=2f256a069ba947c6a6b8cccfea6c7a22)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:10.439514 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc062c9-bc78-4fa3-8fdb-96643be53db9 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:10.442891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 209/414] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:10 2022] GET /identity/v3/projects/2f256a069ba947c6a6b8cccfea6c7a22 => generated 344 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:10.449150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0a5ba-0ba3-47e8-a1dc-6681197c37f5 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:10.587497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0a5ba-0ba3-47e8-a1dc-6681197c37f5 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:10.621814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24004-cb0a-4e63-b5a6-95dcdad01ec6 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:10.622212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24004-cb0a-4e63-b5a6-95dcdad01ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:10.622701 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24004-cb0a-4e63-b5a6-95dcdad01ec6 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:10.625876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 210/415] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:10.632455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351896-277f-4cba-99ff-a72ddf65fbd8 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:10.632955 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351896-277f-4cba-99ff-a72ddf65fbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:10.633441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351896-277f-4cba-99ff-a72ddf65fbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:10.666035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b631d55-2461-4f34-8d6a-ab2794bab0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:10.668822 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b631d55-2461-4f34-8d6a-ab2794bab0c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:10.679168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b631d55-2461-4f34-8d6a-ab2794bab0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:10.680088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b631d55-2461-4f34-8d6a-ab2794bab0c4 None admin] PATH_INFO: `/v3/projects/2f256a069ba947c6a6b8cccfea6c7a22/groups/f3370f3537e04f36bca6f3d3b95c79b0/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:10.684829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b631d55-2461-4f34-8d6a-ab2794bab0c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f256a069ba947c6a6b8cccfea6c7a22, group_id=f3370f3537e04f36bca6f3d3b95c79b0, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:10.688671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b631d55-2461-4f34-8d6a-ab2794bab0c4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:10.703297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-32351896-277f-4cba-99ff-a72ddf65fbd8 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:10.706987 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-32351896-277f-4cba-99ff-a72ddf65fbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:10.728974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 206/416] 10.222.0.61 () {58 vars in 1442 bytes} [Wed Sep 7 13:00:10 2022] PUT /identity/v3/projects/2f256a069ba947c6a6b8cccfea6c7a22/groups/f3370f3537e04f36bca6f3d3b95c79b0/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:00:10.876166 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 211/417] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:00:11.376839 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728459e9-d134-4524-964c-c23e23283284 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:11.377969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728459e9-d134-4524-964c-c23e23283284 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:11.377969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728459e9-d134-4524-964c-c23e23283284 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:11.379176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 207/418] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:11.387252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4652d4-8181-4cd6-b3ce-b088c7328fd1 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:11.388070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4652d4-8181-4cd6-b3ce-b088c7328fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:11.388197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4652d4-8181-4cd6-b3ce-b088c7328fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:11.426107 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-cf4652d4-8181-4cd6-b3ce-b088c7328fd1 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:11.428489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-cf4652d4-8181-4cd6-b3ce-b088c7328fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:11.597793 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 212/419] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:11.655988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728459e9-d134-4524-964c-c23e23283284 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:11.788360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:11.873252 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728459e9-d134-4524-964c-c23e23283284 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:11.924881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea489e-49d8-4e42-8b1f-4b49c78e654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:11.926306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea489e-49d8-4e42-8b1f-4b49c78e654b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:11.927202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea489e-49d8-4e42-8b1f-4b49c78e654b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:11.928099 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea489e-49d8-4e42-8b1f-4b49c78e654b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:11.950969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea489e-49d8-4e42-8b1f-4b49c78e654b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-35ea489e-49d8-4e42-8b1f-4b49c78e654b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:11.961848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:11.963652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:11.965596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:11.965596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:11.965596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:11.965596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:11.965596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:11.965596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:11.965596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:11.965596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:11.965596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:11.965596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:11.965596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:00:11.965596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:11.970380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 208/420] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:11.981153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4652d4-8181-4cd6-b3ce-b088c7328fd1 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:12.129982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:12.196927 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4652d4-8181-4cd6-b3ce-b088c7328fd1 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:12.246911 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea489e-49d8-4e42-8b1f-4b49c78e654b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:12.301219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcfdc9-4d2d-4602-b268-feb5a1d96e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:12.301858 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcfdc9-4d2d-4602-b268-feb5a1d96e2f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:12.302139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcfdc9-4d2d-4602-b268-feb5a1d96e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:12.302413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcfdc9-4d2d-4602-b268-feb5a1d96e2f None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:12.303141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcfdc9-4d2d-4602-b268-feb5a1d96e2f None admin] RBAC: Adding query filter params (name=member) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:12.303448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcfdc9-4d2d-4602-b268-feb5a1d96e2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:12.304903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcfdc9-4d2d-4602-b268-feb5a1d96e2f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:12.322237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 213/421] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 13:00:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:12.331211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcfdc9-4d2d-4602-b268-feb5a1d96e2f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:12.488325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea489e-49d8-4e42-8b1f-4b49c78e654b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:12.516728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:12.538034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcfdc9-4d2d-4602-b268-feb5a1d96e2f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:12.572712 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec78058-e5f8-4330-8e95-82e6c8a6c5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:12.573923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec78058-e5f8-4330-8e95-82e6c8a6c5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:12.574344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec78058-e5f8-4330-8e95-82e6c8a6c5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:12.574636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec78058-e5f8-4330-8e95-82e6c8a6c5d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:12.580530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:12.608130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a786b8-3010-44f0-af73-faef1ebba431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:12.609205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a786b8-3010-44f0-af73-faef1ebba431 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:12.609695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a786b8-3010-44f0-af73-faef1ebba431 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:12.610160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a786b8-3010-44f0-af73-faef1ebba431 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:12.615826 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec78058-e5f8-4330-8e95-82e6c8a6c5d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-cec78058-e5f8-4330-8e95-82e6c8a6c5d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:12.621576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:12.623351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:12.624942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:12.624942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:12.624942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:12.624942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:12.624942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:12.624942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:12.624942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:12.624942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:12.624942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:12.624942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:12.624942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:00:12.624942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:12.624942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 209/422] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:00:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:12.632457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec78058-e5f8-4330-8e95-82e6c8a6c5d4 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:12.747694 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a786b8-3010-44f0-af73-faef1ebba431 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:12.750491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a786b8-3010-44f0-af73-faef1ebba431 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:12.869099 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec78058-e5f8-4330-8e95-82e6c8a6c5d4 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:12.947152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a29d3f-eb1f-4ead-aa62-5e58927cfc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:12.953657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 214/423] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 13:00:12.953657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bedf07-d342-43bf-88b9-9a0f85ef102e None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:12.953657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bedf07-d342-43bf-88b9-9a0f85ef102e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:12.953657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bedf07-d342-43bf-88b9-9a0f85ef102e None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:12.953657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 215/424] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:12.955248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a29d3f-eb1f-4ead-aa62-5e58927cfc01 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:12.955782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bedf07-d342-43bf-88b9-9a0f85ef102e None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:12.956185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a29d3f-eb1f-4ead-aa62-5e58927cfc01 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:12.956637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a29d3f-eb1f-4ead-aa62-5e58927cfc01 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:12.958094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a29d3f-eb1f-4ead-aa62-5e58927cfc01 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:12.958884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a29d3f-eb1f-4ead-aa62-5e58927cfc01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:12.962001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a29d3f-eb1f-4ead-aa62-5e58927cfc01 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:12.982760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 210/425] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:12.985505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcb4d4-0278-4bd8-a088-de6cd7b54a90 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:12.986272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcb4d4-0278-4bd8-a088-de6cd7b54a90 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:12.986992 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcb4d4-0278-4bd8-a088-de6cd7b54a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:13.003763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:13.020646 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-5fbcb4d4-0278-4bd8-a088-de6cd7b54a90 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:13.022526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5fbcb4d4-0278-4bd8-a088-de6cd7b54a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:13.092370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:13.176860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 211/426] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:13.179015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcb4d4-0278-4bd8-a088-de6cd7b54a90 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:13.223243 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bedf07-d342-43bf-88b9-9a0f85ef102e None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:13.358468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fd447-3939-4882-af03-a4cac3c5ba0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:13.360596 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fd447-3939-4882-af03-a4cac3c5ba0f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:13.363478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fd447-3939-4882-af03-a4cac3c5ba0f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:13.364333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fd447-3939-4882-af03-a4cac3c5ba0f None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:13.367203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24fd447-3939-4882-af03-a4cac3c5ba0f None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:13.374111 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24fd447-3939-4882-af03-a4cac3c5ba0f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:13.381323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 216/427] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:12 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:13.392825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fd447-3939-4882-af03-a4cac3c5ba0f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:13.519224 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcb4d4-0278-4bd8-a088-de6cd7b54a90 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:13.621642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3daa514-913d-423c-bf1b-e0b8f3522161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:13.622184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3daa514-913d-423c-bf1b-e0b8f3522161 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:13.622488 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3daa514-913d-423c-bf1b-e0b8f3522161 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:13.622745 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3daa514-913d-423c-bf1b-e0b8f3522161 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:13.659607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:13.688954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fd447-3939-4882-af03-a4cac3c5ba0f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:13.723135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:13.802090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06ae04-ff94-4e13-b188-d02921d1b7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:13.803027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06ae04-ff94-4e13-b188-d02921d1b7ca None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:13.803512 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06ae04-ff94-4e13-b188-d02921d1b7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:13.803957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06ae04-ff94-4e13-b188-d02921d1b7ca None admin] PATH_INFO: `/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:13.806033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06ae04-ff94-4e13-b188-d02921d1b7ca None admin] RBAC: Authorizing `identity:get_group(group_id=0be5c6a5a91743e3a1e5a7676f39026e)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:13.810090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06ae04-ff94-4e13-b188-d02921d1b7ca None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:13.813407 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 217/428] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:13 2022] GET /identity/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e => generated 226 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:00:13.814663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06ae04-ff94-4e13-b188-d02921d1b7ca None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:13.851348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3daa514-913d-423c-bf1b-e0b8f3522161 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:13.855842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3daa514-913d-423c-bf1b-e0b8f3522161 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:13.961812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06ae04-ff94-4e13-b188-d02921d1b7ca None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:14.014621 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 212/429] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 837 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:14.016075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ac447-8104-456d-819b-dec0fddb69d9 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:14.016365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ac447-8104-456d-819b-dec0fddb69d9 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:14.016637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ac447-8104-456d-819b-dec0fddb69d9 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:14.018155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 213/430] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:14.018954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ac447-8104-456d-819b-dec0fddb69d9 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:14.031278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c00f6-2647-413e-a0e4-707510b988bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:14.032088 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c00f6-2647-413e-a0e4-707510b988bc None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:14.033224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c00f6-2647-413e-a0e4-707510b988bc None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:14.033697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c00f6-2647-413e-a0e4-707510b988bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:14.050472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c00f6-2647-413e-a0e4-707510b988bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-214c00f6-2647-413e-a0e4-707510b988bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:14.056731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:14.058751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:14.060344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:14.060344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:14.060344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:14.060344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:14.060344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:14.060344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:14.060344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:14.060344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:14.060344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:14.060344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:14.060344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:00:14.060344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:14.060344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 218/431] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:14.062758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffaa5f-5876-4750-8578-8191dd203fe5 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:14.062758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffaa5f-5876-4750-8578-8191dd203fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:14.063180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffaa5f-5876-4750-8578-8191dd203fe5 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:14.065463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 219/432] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:14.066872 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffaa5f-5876-4750-8578-8191dd203fe5 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:14.099868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:14.110203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:14.169720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ac447-8104-456d-819b-dec0fddb69d9 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:14.215747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffaa5f-5876-4750-8578-8191dd203fe5 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:14.229836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54670119-f843-4998-a4cb-c16c21cfadcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:14.230851 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54670119-f843-4998-a4cb-c16c21cfadcd None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:14.231379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54670119-f843-4998-a4cb-c16c21cfadcd None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:14.231856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54670119-f843-4998-a4cb-c16c21cfadcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:14.233275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54670119-f843-4998-a4cb-c16c21cfadcd None admin] RBAC: Adding query filter params (group.id=0be5c6a5a91743e3a1e5a7676f39026e, role.id=86f185569d134d80841142ae8e9d3f47, scope.project.id=81be097728134d76957fdb5755382cc2) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:14.233896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54670119-f843-4998-a4cb-c16c21cfadcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:14.237178 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54670119-f843-4998-a4cb-c16c21cfadcd None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:14.248851 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 214/433] 10.222.0.61 () {56 vars in 1376 bytes} [Wed Sep 7 13:00:14 2022] GET /identity/v3/role_assignments?group.id=0be5c6a5a91743e3a1e5a7676f39026e&scope.project.id=81be097728134d76957fdb5755382cc2&role.id=86f185569d134d80841142ae8e9d3f47 => generated 611 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:14.251051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bb638-8966-4a2d-b791-69604bc3e6e5 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:14.251575 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bb638-8966-4a2d-b791-69604bc3e6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:14.252055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bb638-8966-4a2d-b791-69604bc3e6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:14.263178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1753d9-79f8-40e4-80e7-8da03ae7afe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:14.263856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1753d9-79f8-40e4-80e7-8da03ae7afe5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:14.264132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1753d9-79f8-40e4-80e7-8da03ae7afe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:14.264416 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1753d9-79f8-40e4-80e7-8da03ae7afe5 None admin] PATH_INFO: `/v3/projects/81be097728134d76957fdb5755382cc2` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:14.265747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1753d9-79f8-40e4-80e7-8da03ae7afe5 None admin] RBAC: Authorizing `identity:get_project(project_id=81be097728134d76957fdb5755382cc2)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:14.267472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1753d9-79f8-40e4-80e7-8da03ae7afe5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:14.269304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 220/434] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:14 2022] GET /identity/v3/projects/81be097728134d76957fdb5755382cc2 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:14.276914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb506-804e-4886-8da5-9d74d959714c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:14.276914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb506-804e-4886-8da5-9d74d959714c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:14.276914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb506-804e-4886-8da5-9d74d959714c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:14.290841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b22bb638-8966-4a2d-b791-69604bc3e6e5 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:14.292124 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-deffb506-804e-4886-8da5-9d74d959714c None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:14.292634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b22bb638-8966-4a2d-b791-69604bc3e6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:14.293336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-deffb506-804e-4886-8da5-9d74d959714c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:14.388887 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 221/435] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:14.390247 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffb506-804e-4886-8da5-9d74d959714c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:14.406988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 215/436] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:14.406988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bb638-8966-4a2d-b791-69604bc3e6e5 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:14.423993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:14.571547 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:14.664524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffb506-804e-4886-8da5-9d74d959714c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:14.678517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bb638-8966-4a2d-b791-69604bc3e6e5 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:14.731366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af74b42-92ce-4c00-944b-8a3d6d18a7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:14.734216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af74b42-92ce-4c00-944b-8a3d6d18a7da None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:14.735004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af74b42-92ce-4c00-944b-8a3d6d18a7da None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:14.735779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af74b42-92ce-4c00-944b-8a3d6d18a7da None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:14.735959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a58b3-1fd2-4b68-bacd-fca058b8a2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:14.736846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a58b3-1fd2-4b68-bacd-fca058b8a2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:14.737201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a58b3-1fd2-4b68-bacd-fca058b8a2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:14.737549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a58b3-1fd2-4b68-bacd-fca058b8a2b1 None admin] PATH_INFO: `/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:14.738471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af74b42-92ce-4c00-944b-8a3d6d18a7da None admin] RBAC: Adding query filter params (name=member) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:14.738865 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a58b3-1fd2-4b68-bacd-fca058b8a2b1 None admin] RBAC: Authorizing `identity:get_group(group_id=0be5c6a5a91743e3a1e5a7676f39026e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:14.739019 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af74b42-92ce-4c00-944b-8a3d6d18a7da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:14.740403 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a58b3-1fd2-4b68-bacd-fca058b8a2b1 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:14.742361 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 216/437] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:14 2022] GET /identity/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e => generated 226 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:14.742576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af74b42-92ce-4c00-944b-8a3d6d18a7da None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:14.742918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a58b3-1fd2-4b68-bacd-fca058b8a2b1 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:14.760318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 222/438] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 13:00:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:14.761817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af74b42-92ce-4c00-944b-8a3d6d18a7da None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:14.887705 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:14.930750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a58b3-1fd2-4b68-bacd-fca058b8a2b1 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:14.933605 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af74b42-92ce-4c00-944b-8a3d6d18a7da None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:14.985109 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54373382-8026-46e4-bb3e-ddec2a1dcf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:14.985847 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54373382-8026-46e4-bb3e-ddec2a1dcf8f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:14.986280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54373382-8026-46e4-bb3e-ddec2a1dcf8f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:14.986698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54373382-8026-46e4-bb3e-ddec2a1dcf8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:14.987726 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54373382-8026-46e4-bb3e-ddec2a1dcf8f None admin] RBAC: Adding query filter params (group.id=0be5c6a5a91743e3a1e5a7676f39026e, role.id=649721c72e5d40deabd3512c529ffdf4, scope.project.id=81be097728134d76957fdb5755382cc2) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:14.988140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54373382-8026-46e4-bb3e-ddec2a1dcf8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:14.990288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54373382-8026-46e4-bb3e-ddec2a1dcf8f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:14.990806 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:14.999406 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 223/439] 10.222.0.61 () {56 vars in 1376 bytes} [Wed Sep 7 13:00:14 2022] GET /identity/v3/role_assignments?group.id=0be5c6a5a91743e3a1e5a7676f39026e&scope.project.id=81be097728134d76957fdb5755382cc2&role.id=649721c72e5d40deabd3512c529ffdf4 => generated 611 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:15.000643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54373382-8026-46e4-bb3e-ddec2a1dcf8f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:15.004889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cbf49-ef17-49de-b622-c6cd3d081aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:15.005685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cbf49-ef17-49de-b622-c6cd3d081aec None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:15.006038 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cbf49-ef17-49de-b622-c6cd3d081aec None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:15.006384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2cbf49-ef17-49de-b622-c6cd3d081aec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:15.023318 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2cbf49-ef17-49de-b622-c6cd3d081aec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-1a2cbf49-ef17-49de-b622-c6cd3d081aec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:15.030671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:15.032499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 217/440] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:00:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:15.036162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cbf49-ef17-49de-b622-c6cd3d081aec None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:15.172239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54373382-8026-46e4-bb3e-ddec2a1dcf8f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:15.202378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2cbf49-ef17-49de-b622-c6cd3d081aec None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:15.220485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b503b0-677c-4118-9177-3458456c07ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:15.221257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b503b0-677c-4118-9177-3458456c07ae None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:15.221651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b503b0-677c-4118-9177-3458456c07ae None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:15.221982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b503b0-677c-4118-9177-3458456c07ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:15.234402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:15.238958 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b503b0-677c-4118-9177-3458456c07ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-c8b503b0-677c-4118-9177-3458456c07ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:15.248853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:15.249902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:15.250696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:15.250696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:15.250696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:15.250696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:15.250696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:15.250696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:15.250696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:15.250696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:15.250696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:15.250696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:15.250696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:00:15.250696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:15.261914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 224/441] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:00:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:15.261914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b503b0-677c-4118-9177-3458456c07ae None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:15.262312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc888ec-9652-4464-abc4-ea2a9b0ae5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:15.262312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc888ec-9652-4464-abc4-ea2a9b0ae5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:15.262312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc888ec-9652-4464-abc4-ea2a9b0ae5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:15.262312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc888ec-9652-4464-abc4-ea2a9b0ae5a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:15.345318 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:15.371943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc888ec-9652-4464-abc4-ea2a9b0ae5a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:15.374417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc888ec-9652-4464-abc4-ea2a9b0ae5a9 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:15.414052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b503b0-677c-4118-9177-3458456c07ae None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:15.473421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dea6b-80a8-4f13-84ea-e350b16fe1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:15.473870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dea6b-80a8-4f13-84ea-e350b16fe1da None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:15.474672 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dea6b-80a8-4f13-84ea-e350b16fe1da None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:15.474944 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dea6b-80a8-4f13-84ea-e350b16fe1da None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:15.475746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dea6b-80a8-4f13-84ea-e350b16fe1da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:15.476057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dea6b-80a8-4f13-84ea-e350b16fe1da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:15.479046 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dea6b-80a8-4f13-84ea-e350b16fe1da None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:15.491581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 225/442] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:15.492417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dea6b-80a8-4f13-84ea-e350b16fe1da None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:15.550655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 218/443] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:15.550655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc888ec-9652-4464-abc4-ea2a9b0ae5a9 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:15.581390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:15.630823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dea6b-80a8-4f13-84ea-e350b16fe1da None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:15.662569 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:15.670224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d71279-07c6-437b-afe2-b86292460581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:15.670902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d71279-07c6-437b-afe2-b86292460581 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:15.671139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d71279-07c6-437b-afe2-b86292460581 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:15.671415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d71279-07c6-437b-afe2-b86292460581 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:15.672101 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d71279-07c6-437b-afe2-b86292460581 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:15.672407 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d71279-07c6-437b-afe2-b86292460581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:15.673854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d71279-07c6-437b-afe2-b86292460581 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:15.681810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc888ec-9652-4464-abc4-ea2a9b0ae5a9 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:15.683502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 226/444] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:15.684367 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d71279-07c6-437b-afe2-b86292460581 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:15.726019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6f995-d1e8-437f-9678-c45f99a7569a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:15.726545 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6f995-d1e8-437f-9678-c45f99a7569a None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:15.726823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6f995-d1e8-437f-9678-c45f99a7569a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:15.727093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6f995-d1e8-437f-9678-c45f99a7569a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:15.805961 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d71279-07c6-437b-afe2-b86292460581 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:15.807916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6f995-d1e8-437f-9678-c45f99a7569a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:15.810198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6f995-d1e8-437f-9678-c45f99a7569a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:15.839741 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:15.859030 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaed33d-1c20-411f-9b28-c4c169fbb26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:15.860301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaed33d-1c20-411f-9b28-c4c169fbb26c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:15.860788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaed33d-1c20-411f-9b28-c4c169fbb26c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:15.861252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaed33d-1c20-411f-9b28-c4c169fbb26c None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:15.863369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaed33d-1c20-411f-9b28-c4c169fbb26c None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:15.866105 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaed33d-1c20-411f-9b28-c4c169fbb26c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:15.869466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 227/445] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:15 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:15.870885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaed33d-1c20-411f-9b28-c4c169fbb26c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:15.911213 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:15.930657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 219/446] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:15.931371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6f995-d1e8-437f-9678-c45f99a7569a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:16.017941 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaed33d-1c20-411f-9b28-c4c169fbb26c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:16.062538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6f995-d1e8-437f-9678-c45f99a7569a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:16.064698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff891226-b863-4435-b9c5-00ff502a4196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:16.065663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff891226-b863-4435-b9c5-00ff502a4196 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:16.066111 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff891226-b863-4435-b9c5-00ff502a4196 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:16.066571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff891226-b863-4435-b9c5-00ff502a4196 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:16.107541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08cf38-62ce-4036-9008-eac984ad2fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:16.108369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08cf38-62ce-4036-9008-eac984ad2fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:16.108737 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08cf38-62ce-4036-9008-eac984ad2fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:16.109099 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08cf38-62ce-4036-9008-eac984ad2fd3 None admin] PATH_INFO: `/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:16.110605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08cf38-62ce-4036-9008-eac984ad2fd3 None admin] RBAC: Authorizing `identity:get_group(group_id=0be5c6a5a91743e3a1e5a7676f39026e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:16.112624 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08cf38-62ce-4036-9008-eac984ad2fd3 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:16.114666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 220/447] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:15 2022] GET /identity/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:16.115721 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08cf38-62ce-4036-9008-eac984ad2fd3 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:16.153453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:16.173108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff891226-b863-4435-b9c5-00ff502a4196 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:16.175806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff891226-b863-4435-b9c5-00ff502a4196 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:16.202121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:16.269620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08cf38-62ce-4036-9008-eac984ad2fd3 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:16.317772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f32bac-1d62-44b3-9b0f-6734cab8b755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:16.318665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f32bac-1d62-44b3-9b0f-6734cab8b755 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:16.319138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f32bac-1d62-44b3-9b0f-6734cab8b755 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:16.319566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f32bac-1d62-44b3-9b0f-6734cab8b755 None admin] PATH_INFO: `/v3/projects/2f256a069ba947c6a6b8cccfea6c7a22` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:16.321189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f32bac-1d62-44b3-9b0f-6734cab8b755 None admin] RBAC: Authorizing `identity:get_project(project_id=2f256a069ba947c6a6b8cccfea6c7a22)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:16.322787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f32bac-1d62-44b3-9b0f-6734cab8b755 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:16.324107 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 228/448] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:16.325112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff891226-b863-4435-b9c5-00ff502a4196 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:16.325284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 221/449] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:16 2022] GET /identity/v3/projects/2f256a069ba947c6a6b8cccfea6c7a22 => generated 344 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:16.331690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f32bac-1d62-44b3-9b0f-6734cab8b755 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:16.448397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:16.462253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff891226-b863-4435-b9c5-00ff502a4196 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:16.469917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f32bac-1d62-44b3-9b0f-6734cab8b755 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:16.503298 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:16.516774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1cd5d-eaa3-44ff-8ae2-5d6ee7ceb198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:16.517848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1cd5d-eaa3-44ff-8ae2-5d6ee7ceb198 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:16.518331 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1cd5d-eaa3-44ff-8ae2-5d6ee7ceb198 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:16.518828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1cd5d-eaa3-44ff-8ae2-5d6ee7ceb198 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:16.520349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1cd5d-eaa3-44ff-8ae2-5d6ee7ceb198 None admin] RBAC: Adding query filter params (group.id=0be5c6a5a91743e3a1e5a7676f39026e, role.id=86f185569d134d80841142ae8e9d3f47, scope.project.id=717d2ed44c684799a826e03b360f281b) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:16.520759 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1cd5d-eaa3-44ff-8ae2-5d6ee7ceb198 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:16.523431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1cd5d-eaa3-44ff-8ae2-5d6ee7ceb198 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:16.524385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f09d5e-3f75-4def-ad10-658ec545e4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:16.525409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f09d5e-3f75-4def-ad10-658ec545e4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:16.525850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f09d5e-3f75-4def-ad10-658ec545e4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:16.526343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f09d5e-3f75-4def-ad10-658ec545e4c8 None admin] PATH_INFO: `/v3/projects/717d2ed44c684799a826e03b360f281b` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:16.528236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f09d5e-3f75-4def-ad10-658ec545e4c8 None admin] RBAC: Authorizing `identity:get_project(project_id=717d2ed44c684799a826e03b360f281b)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:16.530451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f09d5e-3f75-4def-ad10-658ec545e4c8 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:16.540850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 222/450] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:00:16 2022] GET /identity/v3/projects/717d2ed44c684799a826e03b360f281b => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:00:16.541120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 229/451] 10.222.0.61 () {56 vars in 1376 bytes} [Wed Sep 7 13:00:16 2022] GET /identity/v3/role_assignments?group.id=0be5c6a5a91743e3a1e5a7676f39026e&scope.project.id=717d2ed44c684799a826e03b360f281b&role.id=86f185569d134d80841142ae8e9d3f47 => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:16.541273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f09d5e-3f75-4def-ad10-658ec545e4c8 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:16.544048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1cd5d-eaa3-44ff-8ae2-5d6ee7ceb198 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:16.701757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f09d5e-3f75-4def-ad10-658ec545e4c8 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:16.739743 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1cd5d-eaa3-44ff-8ae2-5d6ee7ceb198 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:16.760833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac21156-324a-4d12-9ddd-1503bf219038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:16.761858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac21156-324a-4d12-9ddd-1503bf219038 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:16.762265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac21156-324a-4d12-9ddd-1503bf219038 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:16.762718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac21156-324a-4d12-9ddd-1503bf219038 None admin] PATH_INFO: `/v3/groups/f3370f3537e04f36bca6f3d3b95c79b0` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:16.764834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac21156-324a-4d12-9ddd-1503bf219038 None admin] RBAC: Authorizing `identity:get_group(group_id=f3370f3537e04f36bca6f3d3b95c79b0)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:16.767362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac21156-324a-4d12-9ddd-1503bf219038 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:16.770024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 223/452] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:16 2022] GET /identity/v3/groups/f3370f3537e04f36bca6f3d3b95c79b0 => generated 229 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:16.780016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac21156-324a-4d12-9ddd-1503bf219038 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:16.818300 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a238cd1-3c72-4666-8d78-44641916757c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:16.819423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238cd1-3c72-4666-8d78-44641916757c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:16.819875 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238cd1-3c72-4666-8d78-44641916757c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:16.820279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238cd1-3c72-4666-8d78-44641916757c None admin] PATH_INFO: `/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:16.822052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a238cd1-3c72-4666-8d78-44641916757c None admin] RBAC: Authorizing `identity:get_group(group_id=0be5c6a5a91743e3a1e5a7676f39026e)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:16.824903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a238cd1-3c72-4666-8d78-44641916757c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:16.828076 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 230/453] 10.222.0.61 () {56 vars in 1180 bytes} [Wed Sep 7 13:00:16 2022] GET /identity/v3/groups/0be5c6a5a91743e3a1e5a7676f39026e => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:00:16.842186 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a238cd1-3c72-4666-8d78-44641916757c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:16.887007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:16.894047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:16.973391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac21156-324a-4d12-9ddd-1503bf219038 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:17.031967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a238cd1-3c72-4666-8d78-44641916757c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:17.033662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a65d8e-d308-4d0a-b8c7-e3bfc40bc763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:17.034406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a65d8e-d308-4d0a-b8c7-e3bfc40bc763 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:17.034695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a65d8e-d308-4d0a-b8c7-e3bfc40bc763 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:17.034962 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a65d8e-d308-4d0a-b8c7-e3bfc40bc763 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:17.037547 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a65d8e-d308-4d0a-b8c7-e3bfc40bc763 None admin] RBAC: Adding query filter params (group.id=f3370f3537e04f36bca6f3d3b95c79b0, role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=2f256a069ba947c6a6b8cccfea6c7a22) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:17.037547 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a65d8e-d308-4d0a-b8c7-e3bfc40bc763 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:17.037905 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a65d8e-d308-4d0a-b8c7-e3bfc40bc763 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:17.047165 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 224/454] 10.222.0.61 () {56 vars in 1376 bytes} [Wed Sep 7 13:00:16 2022] GET /identity/v3/role_assignments?group.id=f3370f3537e04f36bca6f3d3b95c79b0&scope.project.id=2f256a069ba947c6a6b8cccfea6c7a22&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 611 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:00:17.110022 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071353b-e14f-476a-bc27-de00a14a0bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:17.111127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071353b-e14f-476a-bc27-de00a14a0bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:17.111659 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071353b-e14f-476a-bc27-de00a14a0bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:17.112139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071353b-e14f-476a-bc27-de00a14a0bd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:17.113634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2071353b-e14f-476a-bc27-de00a14a0bd2 None admin] RBAC: Adding query filter params (group.id=0be5c6a5a91743e3a1e5a7676f39026e, role.id=649721c72e5d40deabd3512c529ffdf4, scope.project.id=717d2ed44c684799a826e03b360f281b) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:17.114181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2071353b-e14f-476a-bc27-de00a14a0bd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:17.117250 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2071353b-e14f-476a-bc27-de00a14a0bd2 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:17.127649 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 231/455] 10.222.0.61 () {56 vars in 1376 bytes} [Wed Sep 7 13:00:16 2022] GET /identity/v3/role_assignments?group.id=0be5c6a5a91743e3a1e5a7676f39026e&scope.project.id=717d2ed44c684799a826e03b360f281b&role.id=649721c72e5d40deabd3512c529ffdf4 => generated 611 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:20.555132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec0bb0-779d-480b-8f2e-bbf76aed4184 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:20.556838 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec0bb0-779d-480b-8f2e-bbf76aed4184 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:20.558225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec0bb0-779d-480b-8f2e-bbf76aed4184 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:20.561166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 225/456] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:20.568073 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd7657-d2d0-40a6-afa8-749ff625577a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:20.569530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd7657-d2d0-40a6-afa8-749ff625577a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:20.570292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd7657-d2d0-40a6-afa8-749ff625577a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:20.611629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8ddd7657-d2d0-40a6-afa8-749ff625577a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:20.614160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8ddd7657-d2d0-40a6-afa8-749ff625577a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:20.759588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 232/457] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:00:21.171488 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd3ee6-d8d5-4863-9c73-abb9c605d8df None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:21.172392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd7657-d2d0-40a6-afa8-749ff625577a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:21.172533 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd3ee6-d8d5-4863-9c73-abb9c605d8df None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:21.172980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd3ee6-d8d5-4863-9c73-abb9c605d8df None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:21.175604 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 226/458] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:21.187971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82c450-7e14-403d-83a9-f92e40da9462 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:21.188471 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82c450-7e14-403d-83a9-f92e40da9462 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:21.188961 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82c450-7e14-403d-83a9-f92e40da9462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:21.209185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:21.226514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-cc82c450-7e14-403d-83a9-f92e40da9462 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:21.228613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-cc82c450-7e14-403d-83a9-f92e40da9462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:21.372800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 227/459] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:00:21.468756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd7657-d2d0-40a6-afa8-749ff625577a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:21.521926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29f43d-793a-4282-93bc-9a54cc24c372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:21.522592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29f43d-793a-4282-93bc-9a54cc24c372 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:21.522863 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29f43d-793a-4282-93bc-9a54cc24c372 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:21.523159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29f43d-793a-4282-93bc-9a54cc24c372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:21.536135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29f43d-793a-4282-93bc-9a54cc24c372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:21.537660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29f43d-793a-4282-93bc-9a54cc24c372 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-2e29f43d-793a-4282-93bc-9a54cc24c372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:21.555529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:21.557924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:21.559764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:21.559764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:21.559764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:21.559764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 233/460] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:21.569039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82c450-7e14-403d-83a9-f92e40da9462 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:21.701557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:21.800788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82c450-7e14-403d-83a9-f92e40da9462 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:21.880595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42e0ef-ec63-4dcc-818a-b9e46f7b0f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:21.881573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42e0ef-ec63-4dcc-818a-b9e46f7b0f79 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:21.882035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42e0ef-ec63-4dcc-818a-b9e46f7b0f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:21.885932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42e0ef-ec63-4dcc-818a-b9e46f7b0f79 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:21.885932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42e0ef-ec63-4dcc-818a-b9e46f7b0f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:21.885932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42e0ef-ec63-4dcc-818a-b9e46f7b0f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:21.887070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42e0ef-ec63-4dcc-818a-b9e46f7b0f79 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:21.901118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29f43d-793a-4282-93bc-9a54cc24c372 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:21.904495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 228/461] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:21.910656 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42e0ef-ec63-4dcc-818a-b9e46f7b0f79 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:22.168045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:22.224950 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29f43d-793a-4282-93bc-9a54cc24c372 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:22.257985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42e0ef-ec63-4dcc-818a-b9e46f7b0f79 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:22.294395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:22.325717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1128d-a22e-4273-842d-203ba54c5259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:22.326278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1128d-a22e-4273-842d-203ba54c5259 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:22.326568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1128d-a22e-4273-842d-203ba54c5259 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:22.326842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1128d-a22e-4273-842d-203ba54c5259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:22.340019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f3ccb-54a0-4a47-a7f1-53f29bc30617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:22.341517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f3ccb-54a0-4a47-a7f1-53f29bc30617 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:22.342244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f3ccb-54a0-4a47-a7f1-53f29bc30617 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:22.342928 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f3ccb-54a0-4a47-a7f1-53f29bc30617 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:22.344386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f3ccb-54a0-4a47-a7f1-53f29bc30617 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:22.347451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f3ccb-54a0-4a47-a7f1-53f29bc30617 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:22.353187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1128d-a22e-4273-842d-203ba54c5259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:22.355304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1128d-a22e-4273-842d-203ba54c5259 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-34c1128d-a22e-4273-842d-203ba54c5259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:22.388719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:22.389840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:22.390894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:22.390894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:22.390894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:22.399155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 229/462] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 13:00:21 2022] POST /identity/v3/users => generated 246 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:00:22.399401 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 234/463] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 493 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:22.404251 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f3ccb-54a0-4a47-a7f1-53f29bc30617 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:22.438356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a494ac-9155-4523-ac34-9a8e90894f53 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:22.440191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a494ac-9155-4523-ac34-9a8e90894f53 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:22.441403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a494ac-9155-4523-ac34-9a8e90894f53 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:22.445322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 235/464] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:22.457825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a823475-6ff3-457b-8578-4e23982cc173 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:22.459220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a823475-6ff3-457b-8578-4e23982cc173 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:22.460011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a823475-6ff3-457b-8578-4e23982cc173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:22.508666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8a823475-6ff3-457b-8578-4e23982cc173 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:22.513601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8a823475-6ff3-457b-8578-4e23982cc173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:22.611263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f3ccb-54a0-4a47-a7f1-53f29bc30617 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:22.676701 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 236/465] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:00:22.677932 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89296507-4e79-4e9b-99c9-af55146a308a None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:22.678397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89296507-4e79-4e9b-99c9-af55146a308a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:22.678772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89296507-4e79-4e9b-99c9-af55146a308a None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:22.681289 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 237/466] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:22.686886 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51734f-0c34-40e6-ad6f-60d1fb2beae3 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:22.687344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51734f-0c34-40e6-ad6f-60d1fb2beae3 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:22.687502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51734f-0c34-40e6-ad6f-60d1fb2beae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:22.693097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0877a-7af9-4ce5-9bc9-b7694ecf6e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:22.694559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0877a-7af9-4ce5-9bc9-b7694ecf6e2f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:22.695014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0877a-7af9-4ce5-9bc9-b7694ecf6e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:22.695554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0877a-7af9-4ce5-9bc9-b7694ecf6e2f None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:22.696739 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0877a-7af9-4ce5-9bc9-b7694ecf6e2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:22.697297 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0877a-7af9-4ce5-9bc9-b7694ecf6e2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:22.700156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0877a-7af9-4ce5-9bc9-b7694ecf6e2f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:22.730438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-bc51734f-0c34-40e6-ad6f-60d1fb2beae3 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:22.731361 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 230/467] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:22.732465 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-bc51734f-0c34-40e6-ad6f-60d1fb2beae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:22.732843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0877a-7af9-4ce5-9bc9-b7694ecf6e2f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:22.809783 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:22.833627 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:22.880214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 238/468] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:00:22.990101 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0877a-7af9-4ce5-9bc9-b7694ecf6e2f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:23.082192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a0cde-1885-418a-b95a-3d37977cc09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:23.083309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a0cde-1885-418a-b95a-3d37977cc09a None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:23.083742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a0cde-1885-418a-b95a-3d37977cc09a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:23.084193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a0cde-1885-418a-b95a-3d37977cc09a None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:23.085405 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a0cde-1885-418a-b95a-3d37977cc09a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:23.088824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a0cde-1885-418a-b95a-3d37977cc09a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:23.138424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 231/469] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 13:00:22 2022] POST /identity/v3/users => generated 248 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:00:23.192887 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51734f-0c34-40e6-ad6f-60d1fb2beae3 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:23.317377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a0cde-1885-418a-b95a-3d37977cc09a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:23.468092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a0cde-1885-418a-b95a-3d37977cc09a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:23.475697 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:23.496619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51734f-0c34-40e6-ad6f-60d1fb2beae3 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:23.521340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b5ff3-7b0a-49a3-b9f3-3d91cd4aef05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:23.521999 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b5ff3-7b0a-49a3-b9f3-3d91cd4aef05 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:23.522294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b5ff3-7b0a-49a3-b9f3-3d91cd4aef05 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:23.522573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b5ff3-7b0a-49a3-b9f3-3d91cd4aef05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:23.537617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b5ff3-7b0a-49a3-b9f3-3d91cd4aef05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:23.539508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b5ff3-7b0a-49a3-b9f3-3d91cd4aef05 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-796b5ff3-7b0a-49a3-b9f3-3d91cd4aef05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:23.558342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:23.559379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:23.560277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:23.560277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:23.560277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:23.562546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 232/470] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:23.564150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cdc1f-d174-4155-89fb-76f6fc298f18 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:23.564616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cdc1f-d174-4155-89fb-76f6fc298f18 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:23.565062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cdc1f-d174-4155-89fb-76f6fc298f18 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:23.566969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 233/471] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:23.573545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f96ccb-873f-442c-ac65-f4f17541321d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:23.573775 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f96ccb-873f-442c-ac65-f4f17541321d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:23.575594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f96ccb-873f-442c-ac65-f4f17541321d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:23.576187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f96ccb-873f-442c-ac65-f4f17541321d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:23.576675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d6bfd-24fe-4da1-ac6d-d34fb6ccddf2 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:23.577167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d6bfd-24fe-4da1-ac6d-d34fb6ccddf2 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:23.577808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d6bfd-24fe-4da1-ac6d-d34fb6ccddf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:23.596122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f96ccb-873f-442c-ac65-f4f17541321d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:23.597757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f96ccb-873f-442c-ac65-f4f17541321d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:23.600338 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b08d6bfd-24fe-4da1-ac6d-d34fb6ccddf2 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:23.601882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b08d6bfd-24fe-4da1-ac6d-d34fb6ccddf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-38f96ccb-873f-442c-ac65-f4f17541321d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:23.616870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:23.617873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:23.618715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:23.618715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:23.618715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:23.624102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 239/472] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:23.625113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f96ccb-873f-442c-ac65-f4f17541321d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:23.667654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:23.691260 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 234/473] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:00:23.692384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d6bfd-24fe-4da1-ac6d-d34fb6ccddf2 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:23.761257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f96ccb-873f-442c-ac65-f4f17541321d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:23.773235 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:23.802317 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a6b89-15d8-4793-b429-bd6f0f337871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:23.802946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a6b89-15d8-4793-b429-bd6f0f337871 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:23.803262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a6b89-15d8-4793-b429-bd6f0f337871 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:23.803540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a6b89-15d8-4793-b429-bd6f0f337871 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:23.804245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a6b89-15d8-4793-b429-bd6f0f337871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:23.804521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a6b89-15d8-4793-b429-bd6f0f337871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:23.805961 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a6b89-15d8-4793-b429-bd6f0f337871 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:23.818467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d6bfd-24fe-4da1-ac6d-d34fb6ccddf2 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:23.820758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 240/474] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:23.827016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a6b89-15d8-4793-b429-bd6f0f337871 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:23.897639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5783d-e567-431e-adb5-a947eb76a21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:23.898095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5783d-e567-431e-adb5-a947eb76a21a None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:23.898332 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5783d-e567-431e-adb5-a947eb76a21a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:23.898563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5783d-e567-431e-adb5-a947eb76a21a None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:23.899230 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5783d-e567-431e-adb5-a947eb76a21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:23.899541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5783d-e567-431e-adb5-a947eb76a21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:23.901561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5783d-e567-431e-adb5-a947eb76a21a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:23.936242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 235/475] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:23.944350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5783d-e567-431e-adb5-a947eb76a21a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:24.029503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:24.044392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a6b89-15d8-4793-b429-bd6f0f337871 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:24.136902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf627e5-dfba-4116-b5e1-5b1562e5c1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:24.137549 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf627e5-dfba-4116-b5e1-5b1562e5c1bc None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:24.137832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf627e5-dfba-4116-b5e1-5b1562e5c1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:24.138108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf627e5-dfba-4116-b5e1-5b1562e5c1bc None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:24.138790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf627e5-dfba-4116-b5e1-5b1562e5c1bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:24.147336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf627e5-dfba-4116-b5e1-5b1562e5c1bc None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:24.185748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 241/476] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 13:00:23 2022] POST /identity/v3/users => generated 249 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:00:24.209555 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf627e5-dfba-4116-b5e1-5b1562e5c1bc None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:24.212865 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5783d-e567-431e-adb5-a947eb76a21a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:24.301320 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:24.305977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f75f1-b06d-4c55-835f-f8d80d68fbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:24.307561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f75f1-b06d-4c55-835f-f8d80d68fbe3 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:24.311139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f75f1-b06d-4c55-835f-f8d80d68fbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:24.311139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f75f1-b06d-4c55-835f-f8d80d68fbe3 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:24.311139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f75f1-b06d-4c55-835f-f8d80d68fbe3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:24.312725 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f75f1-b06d-4c55-835f-f8d80d68fbe3 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:24.366016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 236/477] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 13:00:23 2022] POST /identity/v3/users => generated 248 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:00:24.532962 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf627e5-dfba-4116-b5e1-5b1562e5c1bc None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:24.607592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee739a-0b62-4295-b14c-be25fb881f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:24.608140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee739a-0b62-4295-b14c-be25fb881f61 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:24.608421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee739a-0b62-4295-b14c-be25fb881f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:24.608837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee739a-0b62-4295-b14c-be25fb881f61 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:24.609411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee739a-0b62-4295-b14c-be25fb881f61 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:24.611018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee739a-0b62-4295-b14c-be25fb881f61 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:24.620368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.sql.core [None req-88ee739a-0b62-4295-b14c-be25fb881f61 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 13:00:24.620368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 13:00:24.620368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [parameters: {'id': '14d0c8f18fdc4b69a57c9b439c928aac', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 13:00:24.620368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81795) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-88ee739a-0b62-4295-b14c-be25fb881f61 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application conn.query(q) Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application result.read() Sep 07 13:00:24.643349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return ref.to_dict() Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 13:00:24.645052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application next(self.gen) Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application yield resource Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application next(self.gen) Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise self.value Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application session.commit() Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 13:00:24.646795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self.session.flush() Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application compat.raise_( Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 13:00:24.654519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 13:00:24.660087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application conn.query(q) Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application result.read() Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 13:00:24.664214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [parameters: {'id': '14d0c8f18fdc4b69a57c9b439c928aac', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:24.668410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 13:00:24.671888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:24.676781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 242/478] 10.222.0.61 () {60 vars in 1131 bytes} [Wed Sep 7 13:00:24 2022] POST /identity/v3/roles => generated 144 bytes in 436 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:24.676934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f75f1-b06d-4c55-835f-f8d80d68fbe3 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:24.851626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f75f1-b06d-4c55-835f-f8d80d68fbe3 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:24.903285 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f23c1e-ade2-474b-8069-45825c3cdf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:24.903889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f23c1e-ade2-474b-8069-45825c3cdf2c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:24.904172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f23c1e-ade2-474b-8069-45825c3cdf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:24.904447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f23c1e-ade2-474b-8069-45825c3cdf2c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:24.920698 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f23c1e-ade2-474b-8069-45825c3cdf2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-69f23c1e-ade2-474b-8069-45825c3cdf2c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:24.926346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:24.928493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:24.930098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:24.930098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:24.930098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:24.930098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:24.930098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:24.930098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:24.930098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:24.930098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:24.930098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:24.930098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:24.930098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:00:24.930098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:24.930098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 237/479] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:00:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:24.938387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee739a-0b62-4295-b14c-be25fb881f61 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:24.962338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:25.102228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee739a-0b62-4295-b14c-be25fb881f61 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:25.158371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a25530-b160-4d64-b910-94dacf70473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:25.159422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a25530-b160-4d64-b910-94dacf70473c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:25.159817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a25530-b160-4d64-b910-94dacf70473c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:25.160303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a25530-b160-4d64-b910-94dacf70473c None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:25.161292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a25530-b160-4d64-b910-94dacf70473c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:25.161607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a25530-b160-4d64-b910-94dacf70473c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:25.163572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a25530-b160-4d64-b910-94dacf70473c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:25.180897 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 243/480] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:26.229734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee6ac4-3973-4d66-bcc6-dab68c94aa4e None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:26.230556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee6ac4-3973-4d66-bcc6-dab68c94aa4e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:26.230556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee6ac4-3973-4d66-bcc6-dab68c94aa4e None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:26.232000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 238/481] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:26.246860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289d353-2d68-470c-8992-6f72ba36b4af None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:26.247696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289d353-2d68-470c-8992-6f72ba36b4af None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:26.247696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289d353-2d68-470c-8992-6f72ba36b4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:26.276481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-b289d353-2d68-470c-8992-6f72ba36b4af None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:26.277861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b289d353-2d68-470c-8992-6f72ba36b4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:26.390032 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 244/482] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:26.916847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d91ab-0dfc-4f3e-9865-c08cff8876d7 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:26.917140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d91ab-0dfc-4f3e-9865-c08cff8876d7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:26.917479 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d91ab-0dfc-4f3e-9865-c08cff8876d7 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:26.919211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 239/483] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:26.926801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042e81c-a7a8-4595-bfe8-2aa9d97b5df6 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:26.927243 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042e81c-a7a8-4595-bfe8-2aa9d97b5df6 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:26.927425 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042e81c-a7a8-4595-bfe8-2aa9d97b5df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:26.945043 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d91ab-0dfc-4f3e-9865-c08cff8876d7 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:26.945759 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:26.969988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-6042e81c-a7a8-4595-bfe8-2aa9d97b5df6 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:26.971251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6042e81c-a7a8-4595-bfe8-2aa9d97b5df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:27.040715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:27.145947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 245/484] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:27.163183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d91ab-0dfc-4f3e-9865-c08cff8876d7 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:27.203293 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669e730-3b29-4ca5-9668-dec5ba3b52a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:27.203862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669e730-3b29-4ca5-9668-dec5ba3b52a6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:27.204147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669e730-3b29-4ca5-9668-dec5ba3b52a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:27.204423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669e730-3b29-4ca5-9668-dec5ba3b52a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:27.217462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669e730-3b29-4ca5-9668-dec5ba3b52a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-e669e730-3b29-4ca5-9668-dec5ba3b52a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:27.224053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:27.226485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:27.228163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:27.228163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:27.228163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:27.228163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:27.228163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:27.228163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:27.228163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:27.228163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:27.228163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:27.228163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:27.228163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:27.228163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:27.228163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 240/485] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:27.233642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042e81c-a7a8-4595-bfe8-2aa9d97b5df6 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:27.396018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042e81c-a7a8-4595-bfe8-2aa9d97b5df6 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:27.445531 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888aac2-8fa7-4642-a966-160587689cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:27.446171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888aac2-8fa7-4642-a966-160587689cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:27.446454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888aac2-8fa7-4642-a966-160587689cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:27.446692 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888aac2-8fa7-4642-a966-160587689cb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:27.447466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888aac2-8fa7-4642-a966-160587689cb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:27.447850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888aac2-8fa7-4642-a966-160587689cb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:27.449313 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888aac2-8fa7-4642-a966-160587689cb4 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:27.456654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:27.471264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 246/486] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:27.472469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669e730-3b29-4ca5-9668-dec5ba3b52a6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:27.607190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669e730-3b29-4ca5-9668-dec5ba3b52a6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:27.609113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:27.649335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d6127-f909-4bdb-aba8-c415e2f90397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:27.649995 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d6127-f909-4bdb-aba8-c415e2f90397 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:27.650283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d6127-f909-4bdb-aba8-c415e2f90397 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:27.650706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d6127-f909-4bdb-aba8-c415e2f90397 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:27.663557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d6127-f909-4bdb-aba8-c415e2f90397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:27.665056 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d6127-f909-4bdb-aba8-c415e2f90397 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-7b8d6127-f909-4bdb-aba8-c415e2f90397 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:27.682142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:27.684238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:27.686043 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:27.686043 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:27.686043 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:27.686043 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 241/487] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:27.689722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888aac2-8fa7-4642-a966-160587689cb4 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:27.928292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888aac2-8fa7-4642-a966-160587689cb4 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:28.018228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d6127-f909-4bdb-aba8-c415e2f90397 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:28.021247 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1b844-b109-47ce-b532-c056ae03115c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:28.022879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1b844-b109-47ce-b532-c056ae03115c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:28.023710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1b844-b109-47ce-b532-c056ae03115c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:28.024491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1b844-b109-47ce-b532-c056ae03115c None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:28.026023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1b844-b109-47ce-b532-c056ae03115c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:28.026864 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1b844-b109-47ce-b532-c056ae03115c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:28.029915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1b844-b109-47ce-b532-c056ae03115c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:28.091047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 247/488] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:28.094209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1b844-b109-47ce-b532-c056ae03115c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:28.233901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:28.296711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d6127-f909-4bdb-aba8-c415e2f90397 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:28.346478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1b844-b109-47ce-b532-c056ae03115c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:28.391419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3acf6-f7c9-4b2b-adb5-33817a46613d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:28.392086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3acf6-f7c9-4b2b-adb5-33817a46613d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:28.392369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3acf6-f7c9-4b2b-adb5-33817a46613d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:28.392644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3acf6-f7c9-4b2b-adb5-33817a46613d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:28.401462 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3acf6-f7c9-4b2b-adb5-33817a46613d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:28.402976 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3acf6-f7c9-4b2b-adb5-33817a46613d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-12f3acf6-f7c9-4b2b-adb5-33817a46613d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:28.418948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:28.421213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:28.424029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:28.424029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:28.424029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:28.424029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:28.424029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:28.424029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:28.424029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:28.424029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:28.424029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:00:28.424029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:28.424029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 248/489] 10.222.0.61 () {56 vars in 1093 bytes} [Wed Sep 7 13:00:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:28.431140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3acf6-f7c9-4b2b-adb5-33817a46613d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:28.447245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56987458-67ee-4263-b9bd-90a6f4dd8c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:28.448346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56987458-67ee-4263-b9bd-90a6f4dd8c6d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:28.448846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56987458-67ee-4263-b9bd-90a6f4dd8c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:28.449322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56987458-67ee-4263-b9bd-90a6f4dd8c6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:28.475793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56987458-67ee-4263-b9bd-90a6f4dd8c6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-56987458-67ee-4263-b9bd-90a6f4dd8c6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:28.488062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:28.491570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 242/490] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 473 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:28.495060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b4813-7ad8-4c9a-aae4-317e6d663a2f None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:28.498134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b4813-7ad8-4c9a-aae4-317e6d663a2f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:28.498134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b4813-7ad8-4c9a-aae4-317e6d663a2f None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:28.503573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 243/491] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:28.505199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b4813-7ad8-4c9a-aae4-317e6d663a2f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:28.540802 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:28.554789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3acf6-f7c9-4b2b-adb5-33817a46613d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:28.577879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:28.599585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2186eef-5aa6-4342-b8fc-5b184802249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:28.600225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2186eef-5aa6-4342-b8fc-5b184802249a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:28.600505 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2186eef-5aa6-4342-b8fc-5b184802249a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:28.600821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2186eef-5aa6-4342-b8fc-5b184802249a None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:28.607155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2186eef-5aa6-4342-b8fc-5b184802249a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:28.607155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2186eef-5aa6-4342-b8fc-5b184802249a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:28.607155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2186eef-5aa6-4342-b8fc-5b184802249a None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:28.648356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 249/492] 10.222.0.61 () {56 vars in 1133 bytes} [Wed Sep 7 13:00:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:00:28.655658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d275f9-28f2-4933-a7c7-76303e6f7900 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:28.655946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d275f9-28f2-4933-a7c7-76303e6f7900 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:28.656210 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d275f9-28f2-4933-a7c7-76303e6f7900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:28.677942 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-38d275f9-28f2-4933-a7c7-76303e6f7900 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:28.679570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-38d275f9-28f2-4933-a7c7-76303e6f7900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:28.741595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b4813-7ad8-4c9a-aae4-317e6d663a2f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:28.788406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7c11a-fa8a-4f84-9e70-230c770fc77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:28.789037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7c11a-fa8a-4f84-9e70-230c770fc77c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:28.789310 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7c11a-fa8a-4f84-9e70-230c770fc77c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:28.789580 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7c11a-fa8a-4f84-9e70-230c770fc77c None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:28.790259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7c11a-fa8a-4f84-9e70-230c770fc77c None admin] RBAC: Adding query filter params (name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:28.790561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7c11a-fa8a-4f84-9e70-230c770fc77c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:28.792243 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7c11a-fa8a-4f84-9e70-230c770fc77c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:28.803385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 244/493] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:28.803655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 250/494] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:28.804043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d275f9-28f2-4933-a7c7-76303e6f7900 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:28.807347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25ef7c-b80f-455b-b8e3-d97251759b57 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:28.808007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25ef7c-b80f-455b-b8e3-d97251759b57 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:28.808562 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25ef7c-b80f-455b-b8e3-d97251759b57 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:28.811784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 245/495] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:28.814169 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78e05c-a900-43ba-8a3d-0e7d1887f117 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:28.814746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78e05c-a900-43ba-8a3d-0e7d1887f117 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:28.815366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78e05c-a900-43ba-8a3d-0e7d1887f117 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:28.818473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 246/496] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:28.820326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78e05c-a900-43ba-8a3d-0e7d1887f117 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:28.925305 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:28.958968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d275f9-28f2-4933-a7c7-76303e6f7900 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:28.962650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78e05c-a900-43ba-8a3d-0e7d1887f117 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:28.970597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:29.020454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac7d9-9630-420f-9e0a-67ae45ddd4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:29.021028 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63b5a3-77dc-4c43-a43a-a4a5cc1dd6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:29.021351 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac7d9-9630-420f-9e0a-67ae45ddd4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:29.021789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac7d9-9630-420f-9e0a-67ae45ddd4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:29.022059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63b5a3-77dc-4c43-a43a-a4a5cc1dd6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:29.022231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac7d9-9630-420f-9e0a-67ae45ddd4e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:29.022498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63b5a3-77dc-4c43-a43a-a4a5cc1dd6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:29.022927 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63b5a3-77dc-4c43-a43a-a4a5cc1dd6d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:29.044190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63b5a3-77dc-4c43-a43a-a4a5cc1dd6d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:29.046882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63b5a3-77dc-4c43-a43a-a4a5cc1dd6d7 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-ab63b5a3-77dc-4c43-a43a-a4a5cc1dd6d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:29.071021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:29.072060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:29.073055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:29.073055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:29.073055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:29.075531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 247/497] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:29.076965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e283b4f-1fdf-485e-9a8e-69d65c161224 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:29.077286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e283b4f-1fdf-485e-9a8e-69d65c161224 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:29.077580 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e283b4f-1fdf-485e-9a8e-69d65c161224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:29.108741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-9e283b4f-1fdf-485e-9a8e-69d65c161224 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:29.109974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9e283b4f-1fdf-485e-9a8e-69d65c161224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:29.139422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ac7d9-9630-420f-9e0a-67ae45ddd4e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:29.141626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ac7d9-9630-420f-9e0a-67ae45ddd4e2 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:29.216005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 248/498] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:29.217572 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bce767-3742-41f3-8e43-9b1fdabb607d None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:29.218029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bce767-3742-41f3-8e43-9b1fdabb607d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:29.218468 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bce767-3742-41f3-8e43-9b1fdabb607d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:29.239486 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a8bce767-3742-41f3-8e43-9b1fdabb607d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:29.240683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a8bce767-3742-41f3-8e43-9b1fdabb607d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:29.301596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:29.318965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 249/499] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:29.319734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bce767-3742-41f3-8e43-9b1fdabb607d None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:29.333240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 251/500] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:29.334248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac7d9-9630-420f-9e0a-67ae45ddd4e2 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:29.346088 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:29.508658 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bce767-3742-41f3-8e43-9b1fdabb607d None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:29.516859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac7d9-9630-420f-9e0a-67ae45ddd4e2 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:29.555697 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab0a9b-6148-41ac-9794-e0d7d07efcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:29.556220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab0a9b-6148-41ac-9794-e0d7d07efcc7 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:29.556555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab0a9b-6148-41ac-9794-e0d7d07efcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:29.556847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab0a9b-6148-41ac-9794-e0d7d07efcc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:29.557518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab0a9b-6148-41ac-9794-e0d7d07efcc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:29.557825 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab0a9b-6148-41ac-9794-e0d7d07efcc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:29.559289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab0a9b-6148-41ac-9794-e0d7d07efcc7 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:29.566746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9aad85-bc37-41e3-a4d5-e4be2fbd902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:29.567771 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9aad85-bc37-41e3-a4d5-e4be2fbd902a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:29.568288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9aad85-bc37-41e3-a4d5-e4be2fbd902a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:29.568763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9aad85-bc37-41e3-a4d5-e4be2fbd902a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:29.576536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 250/501] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:29.578999 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab0a9b-6148-41ac-9794-e0d7d07efcc7 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:29.589824 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9aad85-bc37-41e3-a4d5-e4be2fbd902a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-ce9aad85-bc37-41e3-a4d5-e4be2fbd902a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:29.597203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:29.598956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:29.601113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:29.601113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:29.601113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:29.601113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:29.601113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:29.601113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:29.601113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:29.601113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:29.601113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:29.601113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:29.601113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:29.601113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:29.601113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 252/502] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:29.604010 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9aad85-bc37-41e3-a4d5-e4be2fbd902a None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:29.646097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:29.680718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:29.710364 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab0a9b-6148-41ac-9794-e0d7d07efcc7 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:29.720301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9aad85-bc37-41e3-a4d5-e4be2fbd902a None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:29.753132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb81eba-bc4f-49e8-b35f-d27a6c143b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:29.753930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb81eba-bc4f-49e8-b35f-d27a6c143b18 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:29.754292 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb81eba-bc4f-49e8-b35f-d27a6c143b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:29.754672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb81eba-bc4f-49e8-b35f-d27a6c143b18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:29.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd34cb-f790-4165-a137-da41e8893f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:29.763971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd34cb-f790-4165-a137-da41e8893f8e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:29.764249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd34cb-f790-4165-a137-da41e8893f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:29.764515 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd34cb-f790-4165-a137-da41e8893f8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:29.768237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb81eba-bc4f-49e8-b35f-d27a6c143b18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:29.769786 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb81eba-bc4f-49e8-b35f-d27a6c143b18 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:29.777643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd34cb-f790-4165-a137-da41e8893f8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-88bd34cb-f790-4165-a137-da41e8893f8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:29.782964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:29.784788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 253/503] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:29.786365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd34cb-f790-4165-a137-da41e8893f8e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-ebb81eba-bc4f-49e8-b35f-d27a6c143b18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:29.787855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:29.789639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:29.792474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:29.792474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:29.792474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:29.793301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 251/504] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:29.794121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb81eba-bc4f-49e8-b35f-d27a6c143b18 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:29.907184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:29.910673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb81eba-bc4f-49e8-b35f-d27a6c143b18 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:29.917525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd34cb-f790-4165-a137-da41e8893f8e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:29.949882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:29.952846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2957c-cacb-4d79-b5b3-e8d50ed6c8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:29.953617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2957c-cacb-4d79-b5b3-e8d50ed6c8dc None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:29.954100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2957c-cacb-4d79-b5b3-e8d50ed6c8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:29.954421 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2957c-cacb-4d79-b5b3-e8d50ed6c8dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:29.962543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b1d53-fc76-4ab2-9cb9-ce3e79afb0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:29.964133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b1d53-fc76-4ab2-9cb9-ce3e79afb0dc None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:29.964518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b1d53-fc76-4ab2-9cb9-ce3e79afb0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:29.964910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b1d53-fc76-4ab2-9cb9-ce3e79afb0dc None admin] PATH_INFO: `/v3/users/glance` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:29.968069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2957c-cacb-4d79-b5b3-e8d50ed6c8dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:29.970532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2957c-cacb-4d79-b5b3-e8d50ed6c8dc None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:29.978724 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b1d53-fc76-4ab2-9cb9-ce3e79afb0dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:29.981535 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b1d53-fc76-4ab2-9cb9-ce3e79afb0dc None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-ddb2957c-cacb-4d79-b5b3-e8d50ed6c8dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:29.987438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:29.989200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:29.990852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:29.990852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:29.990852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:29.990852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 252/505] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:29.990852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2957c-cacb-4d79-b5b3-e8d50ed6c8dc None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-6b0b1d53-fc76-4ab2-9cb9-ce3e79afb0dc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:30.005099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:30.007028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:30.008200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:30.008200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:30.008200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:30.008200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:30.008200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:30.008200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:30.008200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:30.008200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:30.008200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 13:00:30.008200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:30.008200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 254/506] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:30.008917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b1d53-fc76-4ab2-9cb9-ce3e79afb0dc None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:30.142105 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2957c-cacb-4d79-b5b3-e8d50ed6c8dc None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:30.165942 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b1d53-fc76-4ab2-9cb9-ce3e79afb0dc None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:30.192281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143b82f-1224-4138-b0a1-37558776fad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:30.193088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143b82f-1224-4138-b0a1-37558776fad6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:30.193453 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143b82f-1224-4138-b0a1-37558776fad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:30.193813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143b82f-1224-4138-b0a1-37558776fad6 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:30.194584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143b82f-1224-4138-b0a1-37558776fad6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:30.194959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143b82f-1224-4138-b0a1-37558776fad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:30.196662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143b82f-1224-4138-b0a1-37558776fad6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:30.206971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 253/507] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:30.208106 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143b82f-1224-4138-b0a1-37558776fad6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:30.226333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122381ee-db4f-4296-9ae9-9e22cb24f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:30.229164 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122381ee-db4f-4296-9ae9-9e22cb24f214 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:30.229593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122381ee-db4f-4296-9ae9-9e22cb24f214 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:30.230054 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122381ee-db4f-4296-9ae9-9e22cb24f214 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:30.231041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122381ee-db4f-4296-9ae9-9e22cb24f214 None admin] RBAC: Adding query filter params (name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:30.233838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122381ee-db4f-4296-9ae9-9e22cb24f214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:30.235902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122381ee-db4f-4296-9ae9-9e22cb24f214 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:30.238874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:30.249526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 255/508] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:30.250573 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122381ee-db4f-4296-9ae9-9e22cb24f214 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:30.336406 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:30.377645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143b82f-1224-4138-b0a1-37558776fad6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:30.435675 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122381ee-db4f-4296-9ae9-9e22cb24f214 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:30.441234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a3f3c-f027-46ee-a35c-9f772488c8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:30.442303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a3f3c-f027-46ee-a35c-9f772488c8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:30.442879 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a3f3c-f027-46ee-a35c-9f772488c8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:30.443355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a3f3c-f027-46ee-a35c-9f772488c8d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:30.444402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a3f3c-f027-46ee-a35c-9f772488c8d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:30.444912 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a3f3c-f027-46ee-a35c-9f772488c8d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:30.447814 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a3f3c-f027-46ee-a35c-9f772488c8d6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:30.465844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 254/509] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:30.467548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a3f3c-f027-46ee-a35c-9f772488c8d6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:30.495436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d80b2-eac4-4160-b0c9-683eb7ca4603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:30.496420 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d80b2-eac4-4160-b0c9-683eb7ca4603 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:30.496843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d80b2-eac4-4160-b0c9-683eb7ca4603 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:30.497301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d80b2-eac4-4160-b0c9-683eb7ca4603 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:30.498393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d80b2-eac4-4160-b0c9-683eb7ca4603 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:30.498799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d80b2-eac4-4160-b0c9-683eb7ca4603 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:30.501283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d80b2-eac4-4160-b0c9-683eb7ca4603 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:30.523571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 256/510] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:30.525104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d80b2-eac4-4160-b0c9-683eb7ca4603 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:30.598778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:30.654539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a3f3c-f027-46ee-a35c-9f772488c8d6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:30.707291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d80b2-eac4-4160-b0c9-683eb7ca4603 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:30.711290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f26c90-2e41-43c9-a608-148a34832ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:30.712025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f26c90-2e41-43c9-a608-148a34832ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:30.712367 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f26c90-2e41-43c9-a608-148a34832ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:30.712741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f26c90-2e41-43c9-a608-148a34832ca9 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:30.713572 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f26c90-2e41-43c9-a608-148a34832ca9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:30.713954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f26c90-2e41-43c9-a608-148a34832ca9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:30.715553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f26c90-2e41-43c9-a608-148a34832ca9 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:30.719914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:30.753251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cb597-0b40-4ebc-97d7-aba59fd91010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:30.754040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cb597-0b40-4ebc-97d7-aba59fd91010 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:30.754398 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cb597-0b40-4ebc-97d7-aba59fd91010 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:30.754749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cb597-0b40-4ebc-97d7-aba59fd91010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:30.763389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 255/511] 10.222.0.61 () {56 vars in 1137 bytes} [Wed Sep 7 13:00:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:30.764845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f26c90-2e41-43c9-a608-148a34832ca9 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:30.767801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cb597-0b40-4ebc-97d7-aba59fd91010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:30.769508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cb597-0b40-4ebc-97d7-aba59fd91010 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-cc5cb597-0b40-4ebc-97d7-aba59fd91010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:30.794199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:30.795249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:30.796098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:30.796098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:30.796098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:30.796740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 257/512] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:30.798214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cb597-0b40-4ebc-97d7-aba59fd91010 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:30.918719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cb597-0b40-4ebc-97d7-aba59fd91010 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:30.924496 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f26c90-2e41-43c9-a608-148a34832ca9 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:30.958800 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cf72a-f333-4e9c-9b80-e1a7979aad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:30.959384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cf72a-f333-4e9c-9b80-e1a7979aad3c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:30.959663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cf72a-f333-4e9c-9b80-e1a7979aad3c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:30.959932 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cf72a-f333-4e9c-9b80-e1a7979aad3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:30.967772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cf72a-f333-4e9c-9b80-e1a7979aad3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:30.969520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cf72a-f333-4e9c-9b80-e1a7979aad3c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:30.977804 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-8e1cf72a-f333-4e9c-9b80-e1a7979aad3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:30.983813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:30.985449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:30.987211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:30.987211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:30.987211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:30.987211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:30.987211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:30.987211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:30.987211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:30.987211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:30.988081 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f30f2-028d-4890-97ad-1ee32cc53baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:30.988240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 258/513] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:30.988240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cf72a-f333-4e9c-9b80-e1a7979aad3c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:30.988482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f30f2-028d-4890-97ad-1ee32cc53baf None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:30.988482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f30f2-028d-4890-97ad-1ee32cc53baf None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:30.988482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f30f2-028d-4890-97ad-1ee32cc53baf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:31.008537 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f30f2-028d-4890-97ad-1ee32cc53baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:31.011042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f30f2-028d-4890-97ad-1ee32cc53baf None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:31.024374 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-ba7f30f2-028d-4890-97ad-1ee32cc53baf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:31.031401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:31.033206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:31.034790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:31.034790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:31.034790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:31.034790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 256/514] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:31.034790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f30f2-028d-4890-97ad-1ee32cc53baf None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:31.146428 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cf72a-f333-4e9c-9b80-e1a7979aad3c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:31.187575 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728863ae-6988-4566-9b5b-61331199f5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:31.187867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f30f2-028d-4890-97ad-1ee32cc53baf None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:31.188624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728863ae-6988-4566-9b5b-61331199f5b4 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:31.188997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728863ae-6988-4566-9b5b-61331199f5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:31.189362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728863ae-6988-4566-9b5b-61331199f5b4 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:31.190203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728863ae-6988-4566-9b5b-61331199f5b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:31.191978 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728863ae-6988-4566-9b5b-61331199f5b4 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:31.220453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 259/515] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 13:00:30 2022] POST /identity/v3/users => generated 247 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:00:31.221405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728863ae-6988-4566-9b5b-61331199f5b4 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:31.245979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636af28c-1572-4125-8e74-fb40bb222dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:31.246984 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636af28c-1572-4125-8e74-fb40bb222dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:31.247455 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636af28c-1572-4125-8e74-fb40bb222dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:31.247882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636af28c-1572-4125-8e74-fb40bb222dc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:31.324973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:31.357133 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636af28c-1572-4125-8e74-fb40bb222dc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:31.360488 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636af28c-1572-4125-8e74-fb40bb222dc5 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:31.374000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:31.415650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728863ae-6988-4566-9b5b-61331199f5b4 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:31.482690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177fb53-d075-4044-9848-a44f265bd3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:31.483680 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177fb53-d075-4044-9848-a44f265bd3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:31.484282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177fb53-d075-4044-9848-a44f265bd3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:31.484634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177fb53-d075-4044-9848-a44f265bd3d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:31.485878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177fb53-d075-4044-9848-a44f265bd3d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:31.486426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177fb53-d075-4044-9848-a44f265bd3d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:31.490010 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177fb53-d075-4044-9848-a44f265bd3d9 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:31.513587 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 260/516] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:31.519233 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177fb53-d075-4044-9848-a44f265bd3d9 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:31.548700 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 257/517] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:31.549821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636af28c-1572-4125-8e74-fb40bb222dc5 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:31.693562 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:31.697387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177fb53-d075-4044-9848-a44f265bd3d9 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:31.718092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636af28c-1572-4125-8e74-fb40bb222dc5 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:31.739561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccfda0-7b8a-4c62-bca7-729242bd9c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:31.740082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccfda0-7b8a-4c62-bca7-729242bd9c8d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:31.740359 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccfda0-7b8a-4c62-bca7-729242bd9c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:31.740692 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccfda0-7b8a-4c62-bca7-729242bd9c8d None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:31.741535 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccfda0-7b8a-4c62-bca7-729242bd9c8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:31.741962 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccfda0-7b8a-4c62-bca7-729242bd9c8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:31.743785 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccfda0-7b8a-4c62-bca7-729242bd9c8d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:31.750318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:31.765019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54ca60-b4df-44b9-9041-24fafab10b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:31.765660 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54ca60-b4df-44b9-9041-24fafab10b9e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:31.765941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54ca60-b4df-44b9-9041-24fafab10b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:31.766362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54ca60-b4df-44b9-9041-24fafab10b9e None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:31.767452 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54ca60-b4df-44b9-9041-24fafab10b9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:31.767766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54ca60-b4df-44b9-9041-24fafab10b9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:31.769250 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54ca60-b4df-44b9-9041-24fafab10b9e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:31.769695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 261/518] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:31.770907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccfda0-7b8a-4c62-bca7-729242bd9c8d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:31.794448 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 258/519] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:31.796324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54ca60-b4df-44b9-9041-24fafab10b9e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:31.931764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccfda0-7b8a-4c62-bca7-729242bd9c8d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:31.955077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54ca60-b4df-44b9-9041-24fafab10b9e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:31.996084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513af5df-179b-4310-a295-f35c4306b7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:31.996629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513af5df-179b-4310-a295-f35c4306b7e4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:31.996905 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513af5df-179b-4310-a295-f35c4306b7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:31.997174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513af5df-179b-4310-a295-f35c4306b7e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:32.002622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e829f8-4be1-45da-a62d-3d20ca1784bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:32.003779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e829f8-4be1-45da-a62d-3d20ca1784bb None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:32.004275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e829f8-4be1-45da-a62d-3d20ca1784bb None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:32.004747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e829f8-4be1-45da-a62d-3d20ca1784bb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:32.010241 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513af5df-179b-4310-a295-f35c4306b7e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:32.011769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513af5df-179b-4310-a295-f35c4306b7e4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:32.014680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:32.019070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e829f8-4be1-45da-a62d-3d20ca1784bb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:32.021689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e829f8-4be1-45da-a62d-3d20ca1784bb None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-513af5df-179b-4310-a295-f35c4306b7e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:32.029787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:32.030957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:32.032034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:32.032034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:32.032034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:32.035451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 259/520] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:32.036597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513af5df-179b-4310-a295-f35c4306b7e4 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-b4e829f8-4be1-45da-a62d-3d20ca1784bb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:32.045518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:32.046484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:32.047548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:32.047548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:32.047548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:32.047548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:32.047548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:32.047548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:32.047548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:32.047548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:32.047548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:00:32.047548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:32.051538 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 262/521] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:32.053073 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e829f8-4be1-45da-a62d-3d20ca1784bb None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:32.137567 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:32.191735 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513af5df-179b-4310-a295-f35c4306b7e4 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:32.196336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e829f8-4be1-45da-a62d-3d20ca1784bb None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:32.236061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c80a3-4bee-483c-aa43-436f2a645cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:32.236688 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c80a3-4bee-483c-aa43-436f2a645cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:32.237126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c80a3-4bee-483c-aa43-436f2a645cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:32.237230 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c80a3-4bee-483c-aa43-436f2a645cc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:32.237978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c80a3-4bee-483c-aa43-436f2a645cc3 None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=ba93e2d4aaed4c5595d5fd7363626bcb) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:32.238285 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c80a3-4bee-483c-aa43-436f2a645cc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:32.239819 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c80a3-4bee-483c-aa43-436f2a645cc3 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:32.255250 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 260/522] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:00:32 2022] GET /identity/v3/role_assignments?user.id=ba93e2d4aaed4c5595d5fd7363626bcb&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:32.255451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877cf37-fe2e-4c54-898e-af18aa903f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:32.255555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c80a3-4bee-483c-aa43-436f2a645cc3 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:32.255659 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877cf37-fe2e-4c54-898e-af18aa903f8e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:32.255659 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877cf37-fe2e-4c54-898e-af18aa903f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:32.255659 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877cf37-fe2e-4c54-898e-af18aa903f8e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:32.269648 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877cf37-fe2e-4c54-898e-af18aa903f8e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:32.272562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877cf37-fe2e-4c54-898e-af18aa903f8e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-a877cf37-fe2e-4c54-898e-af18aa903f8e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:32.294791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:32.296558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:32.298934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:32.298934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:32.298934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:32.298934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:32.298934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:32.298934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:32.298934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:32.298934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:32.298934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 13:00:32.298934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:32.304007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 263/523] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:32.305572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877cf37-fe2e-4c54-898e-af18aa903f8e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:32.380617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:32.451189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c80a3-4bee-483c-aa43-436f2a645cc3 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:32.494281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ae077-f510-4e45-ac02-ceb421508cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:32.495343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ae077-f510-4e45-ac02-ceb421508cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:32.495602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ae077-f510-4e45-ac02-ceb421508cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:32.495883 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ae077-f510-4e45-ac02-ceb421508cf5 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:32.496588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ae077-f510-4e45-ac02-ceb421508cf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:32.496896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ae077-f510-4e45-ac02-ceb421508cf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:32.498374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ae077-f510-4e45-ac02-ceb421508cf5 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:32.511441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877cf37-fe2e-4c54-898e-af18aa903f8e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:32.519157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 261/524] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:32.519157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ae077-f510-4e45-ac02-ceb421508cf5 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:32.549417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:32.583948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a892c-b9cc-4ea9-8821-8faad8ac58a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:32.585690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a892c-b9cc-4ea9-8821-8faad8ac58a4 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:32.585690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a892c-b9cc-4ea9-8821-8faad8ac58a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:32.585690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a892c-b9cc-4ea9-8821-8faad8ac58a4 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:32.586106 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a892c-b9cc-4ea9-8821-8faad8ac58a4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:32.586347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a892c-b9cc-4ea9-8821-8faad8ac58a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:32.588429 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a892c-b9cc-4ea9-8821-8faad8ac58a4 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:32.612153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 264/525] 10.222.0.61 () {56 vars in 1137 bytes} [Wed Sep 7 13:00:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:32.613055 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a892c-b9cc-4ea9-8821-8faad8ac58a4 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:32.725409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ae077-f510-4e45-ac02-ceb421508cf5 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:32.782437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:32.797977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a892c-b9cc-4ea9-8821-8faad8ac58a4 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:32.818100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95938821-b10b-415a-88bf-56fa7511f74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:32.819428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95938821-b10b-415a-88bf-56fa7511f74e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:32.820051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95938821-b10b-415a-88bf-56fa7511f74e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:32.820713 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95938821-b10b-415a-88bf-56fa7511f74e None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:32.822901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95938821-b10b-415a-88bf-56fa7511f74e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:32.823696 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95938821-b10b-415a-88bf-56fa7511f74e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:32.826615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95938821-b10b-415a-88bf-56fa7511f74e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:32.864841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 262/526] 10.222.0.61 () {56 vars in 1139 bytes} [Wed Sep 7 13:00:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:32.866517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95938821-b10b-415a-88bf-56fa7511f74e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:32.881948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5b98-c88f-4b54-9fde-0e80fea8be26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:32.883398 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5b98-c88f-4b54-9fde-0e80fea8be26 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:32.884084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5b98-c88f-4b54-9fde-0e80fea8be26 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:32.886072 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5b98-c88f-4b54-9fde-0e80fea8be26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:32.904687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc5b98-c88f-4b54-9fde-0e80fea8be26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:32.909400 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc5b98-c88f-4b54-9fde-0e80fea8be26 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-dddc5b98-c88f-4b54-9fde-0e80fea8be26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:32.942915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:32.947893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:32.949860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:32.949860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:32.949860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:32.949860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:32.949860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:32.949860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:32.949860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:32.949860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:32.962472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 265/527] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:32.962472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5b98-c88f-4b54-9fde-0e80fea8be26 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:33.027274 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:33.056010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95938821-b10b-415a-88bf-56fa7511f74e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:33.116801 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac5b5f-ca1b-4a6c-bc00-adb18bf4ac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:33.117533 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac5b5f-ca1b-4a6c-bc00-adb18bf4ac0e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:33.117866 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac5b5f-ca1b-4a6c-bc00-adb18bf4ac0e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:33.118246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac5b5f-ca1b-4a6c-bc00-adb18bf4ac0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:33.151611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5b98-c88f-4b54-9fde-0e80fea8be26 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:33.176414 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:33.205286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac5b5f-ca1b-4a6c-bc00-adb18bf4ac0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:33.206805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac5b5f-ca1b-4a6c-bc00-adb18bf4ac0e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:33.214101 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c2bd2-452c-4b95-9070-f77dd5d6ad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:33.215220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c2bd2-452c-4b95-9070-f77dd5d6ad0b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:33.216106 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c2bd2-452c-4b95-9070-f77dd5d6ad0b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:33.216873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c2bd2-452c-4b95-9070-f77dd5d6ad0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:33.336617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 263/528] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:33.337626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac5b5f-ca1b-4a6c-bc00-adb18bf4ac0e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:33.356831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c2bd2-452c-4b95-9070-f77dd5d6ad0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:33.360931 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c2bd2-452c-4b95-9070-f77dd5d6ad0b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:33.426775 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:33.441813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:33.471868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac5b5f-ca1b-4a6c-bc00-adb18bf4ac0e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:33.516366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16eb5b9-aee8-44dd-b7aa-2010e066f6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:33.517005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16eb5b9-aee8-44dd-b7aa-2010e066f6ff None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:33.517280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16eb5b9-aee8-44dd-b7aa-2010e066f6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:33.517549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16eb5b9-aee8-44dd-b7aa-2010e066f6ff None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:33.518277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16eb5b9-aee8-44dd-b7aa-2010e066f6ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:33.518581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16eb5b9-aee8-44dd-b7aa-2010e066f6ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:33.519992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 266/529] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:33.520127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16eb5b9-aee8-44dd-b7aa-2010e066f6ff None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:33.521305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c2bd2-452c-4b95-9070-f77dd5d6ad0b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:33.553780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 264/530] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:33.553780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16eb5b9-aee8-44dd-b7aa-2010e066f6ff None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:33.652676 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c2bd2-452c-4b95-9070-f77dd5d6ad0b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:33.690076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16eb5b9-aee8-44dd-b7aa-2010e066f6ff None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:33.694475 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e41d03-b142-4b57-9424-2227abd0d64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:33.695553 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e41d03-b142-4b57-9424-2227abd0d64f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:33.696044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e41d03-b142-4b57-9424-2227abd0d64f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:33.696534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e41d03-b142-4b57-9424-2227abd0d64f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:33.699737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:33.711473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e41d03-b142-4b57-9424-2227abd0d64f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:33.713963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e41d03-b142-4b57-9424-2227abd0d64f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-d7e41d03-b142-4b57-9424-2227abd0d64f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:33.732052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:33.733235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:33.734265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:33.734265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:33.734265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:33.740367 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 267/531] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:33.740367 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e41d03-b142-4b57-9424-2227abd0d64f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:33.742281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7b41a-252d-46ab-8f32-3eea0900e78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:33.744094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7b41a-252d-46ab-8f32-3eea0900e78f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:33.744560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7b41a-252d-46ab-8f32-3eea0900e78f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:33.744975 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7b41a-252d-46ab-8f32-3eea0900e78f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:33.753943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:33.761243 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7b41a-252d-46ab-8f32-3eea0900e78f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:33.762956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7b41a-252d-46ab-8f32-3eea0900e78f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-19d7b41a-252d-46ab-8f32-3eea0900e78f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:33.781629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:33.782634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:33.783616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:33.783616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:33.783616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:33.790031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 265/532] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:33.792172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7b41a-252d-46ab-8f32-3eea0900e78f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:33.865540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e41d03-b142-4b57-9424-2227abd0d64f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:33.913952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7b41a-252d-46ab-8f32-3eea0900e78f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:33.931905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcbbd0-3001-4a87-99f7-a768d614a479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:33.933041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcbbd0-3001-4a87-99f7-a768d614a479 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:33.933590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcbbd0-3001-4a87-99f7-a768d614a479 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:33.934180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcbbd0-3001-4a87-99f7-a768d614a479 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:33.935657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcbbd0-3001-4a87-99f7-a768d614a479 None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=b4f82adce75b44c6baf85b6b96dd0728) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:33.936276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcbbd0-3001-4a87-99f7-a768d614a479 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:33.939347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcbbd0-3001-4a87-99f7-a768d614a479 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:33.955220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 268/533] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:00:33 2022] GET /identity/v3/role_assignments?user.id=b4f82adce75b44c6baf85b6b96dd0728&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:33.956588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcbbd0-3001-4a87-99f7-a768d614a479 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:33.973512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7fd22-b5ad-48f2-b24c-c7ebc246ebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:33.974471 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7fd22-b5ad-48f2-b24c-c7ebc246ebec None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:33.974932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7fd22-b5ad-48f2-b24c-c7ebc246ebec None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:33.975412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7fd22-b5ad-48f2-b24c-c7ebc246ebec None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:33.976307 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7fd22-b5ad-48f2-b24c-c7ebc246ebec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:33.976807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7fd22-b5ad-48f2-b24c-c7ebc246ebec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:33.979264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7fd22-b5ad-48f2-b24c-c7ebc246ebec None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:34.006347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 266/534] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:34.030549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7fd22-b5ad-48f2-b24c-c7ebc246ebec None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:34.081155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:34.114972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:34.181489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7fd22-b5ad-48f2-b24c-c7ebc246ebec None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:34.212179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcbbd0-3001-4a87-99f7-a768d614a479 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:34.225636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4d187-83c9-42a9-80ea-aee5d35d40bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:34.226684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4d187-83c9-42a9-80ea-aee5d35d40bd None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:34.227199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4d187-83c9-42a9-80ea-aee5d35d40bd None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:34.227777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4d187-83c9-42a9-80ea-aee5d35d40bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:34.237459 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4d187-83c9-42a9-80ea-aee5d35d40bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:34.239422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4d187-83c9-42a9-80ea-aee5d35d40bd None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-31b4d187-83c9-42a9-80ea-aee5d35d40bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:34.254202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:34.255228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:34.256092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:34.256092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:34.256092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:34.256092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:34.256092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:34.256092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:34.256092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:34.256092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:34.259007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 267/535] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:34.273232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4d187-83c9-42a9-80ea-aee5d35d40bd None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:34.330972 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42fa9d-2b46-413b-b011-693d44149ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:34.332075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42fa9d-2b46-413b-b011-693d44149ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:34.332670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42fa9d-2b46-413b-b011-693d44149ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:34.333141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42fa9d-2b46-413b-b011-693d44149ba2 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:34.334378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42fa9d-2b46-413b-b011-693d44149ba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:34.334913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42fa9d-2b46-413b-b011-693d44149ba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:34.338112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42fa9d-2b46-413b-b011-693d44149ba2 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:34.381521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 269/536] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:34.394966 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42fa9d-2b46-413b-b011-693d44149ba2 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:34.492930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4d187-83c9-42a9-80ea-aee5d35d40bd None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:34.505879 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:34.558690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30995dec-9869-44da-9fb9-66724909d935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:34.559369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30995dec-9869-44da-9fb9-66724909d935 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:34.559647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30995dec-9869-44da-9fb9-66724909d935 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:34.559955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30995dec-9869-44da-9fb9-66724909d935 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:34.560780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30995dec-9869-44da-9fb9-66724909d935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:34.561093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30995dec-9869-44da-9fb9-66724909d935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:34.562603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30995dec-9869-44da-9fb9-66724909d935 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:34.584486 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 268/537] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:34.611033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30995dec-9869-44da-9fb9-66724909d935 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:34.659169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42fa9d-2b46-413b-b011-693d44149ba2 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:34.693014 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:34.749419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6bda8-7557-4dde-84b8-20f70698ca7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:34.750384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6bda8-7557-4dde-84b8-20f70698ca7b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:34.750831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6bda8-7557-4dde-84b8-20f70698ca7b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:34.751327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6bda8-7557-4dde-84b8-20f70698ca7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:34.782138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6bda8-7557-4dde-84b8-20f70698ca7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:34.785165 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6bda8-7557-4dde-84b8-20f70698ca7b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-69b6bda8-7557-4dde-84b8-20f70698ca7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:34.813568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:34.815285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:34.816967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:34.816967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:34.816967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:34.816967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:34.816967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:34.816967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:34.816967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:34.816967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:34.816967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 270/538] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:34.828063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6bda8-7557-4dde-84b8-20f70698ca7b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:34.904052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30995dec-9869-44da-9fb9-66724909d935 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:34.956759 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6bda8-7557-4dde-84b8-20f70698ca7b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:34.974909 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537d3dc-2ae7-4eee-9e09-0b85e2d0d825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:34.976144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537d3dc-2ae7-4eee-9e09-0b85e2d0d825 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:34.976640 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537d3dc-2ae7-4eee-9e09-0b85e2d0d825 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:34.977275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537d3dc-2ae7-4eee-9e09-0b85e2d0d825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:34.978808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537d3dc-2ae7-4eee-9e09-0b85e2d0d825 None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=8356f60a0f684484abc8f8d6fb6a7be6) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:34.979410 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537d3dc-2ae7-4eee-9e09-0b85e2d0d825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:34.982480 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537d3dc-2ae7-4eee-9e09-0b85e2d0d825 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:34.994987 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 269/539] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:00:34 2022] GET /identity/v3/role_assignments?user.id=8356f60a0f684484abc8f8d6fb6a7be6&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 259 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:34.997529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8df4c-e3fb-4fe0-bc91-05b68f02db8e None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:34.998032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8df4c-e3fb-4fe0-bc91-05b68f02db8e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:34.998523 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8df4c-e3fb-4fe0-bc91-05b68f02db8e None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:35.001282 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 270/540] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:35.004661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab1c5a-d5d9-442b-98a9-7d4568c82441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:35.005310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab1c5a-d5d9-442b-98a9-7d4568c82441 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:35.005591 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab1c5a-d5d9-442b-98a9-7d4568c82441 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:35.005863 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab1c5a-d5d9-442b-98a9-7d4568c82441 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:35.006587 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab1c5a-d5d9-442b-98a9-7d4568c82441 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:35.006896 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab1c5a-d5d9-442b-98a9-7d4568c82441 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:35.011632 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a900938-5a0a-43e7-a135-54e6f8cbfd2b None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:35.012129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a900938-5a0a-43e7-a135-54e6f8cbfd2b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:35.012622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a900938-5a0a-43e7-a135-54e6f8cbfd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:35.015239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab1c5a-d5d9-442b-98a9-7d4568c82441 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:35.040094 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 271/541] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:00:35.046218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-9a900938-5a0a-43e7-a135-54e6f8cbfd2b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:35.048464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9a900938-5a0a-43e7-a135-54e6f8cbfd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:35.050892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab1c5a-d5d9-442b-98a9-7d4568c82441 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:35.099788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:35.111222 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:35.204754 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 271/542] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:00:35.307013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab1c5a-d5d9-442b-98a9-7d4568c82441 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:35.381709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f89ccb-3cb2-4d7c-b4f2-f10249795133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:35.382710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f89ccb-3cb2-4d7c-b4f2-f10249795133 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:35.383223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f89ccb-3cb2-4d7c-b4f2-f10249795133 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:35.383747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f89ccb-3cb2-4d7c-b4f2-f10249795133 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:35.385684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f89ccb-3cb2-4d7c-b4f2-f10249795133 None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=de05c08e30dd4fff93f1c97aac9553d2) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:35.386213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f89ccb-3cb2-4d7c-b4f2-f10249795133 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:35.389013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f89ccb-3cb2-4d7c-b4f2-f10249795133 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:35.399574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 272/543] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:00:35 2022] GET /identity/v3/role_assignments?user.id=de05c08e30dd4fff93f1c97aac9553d2&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 259 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:00:35.771392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66084e-1faa-477a-80d5-48713fd96670 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:35.771818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66084e-1faa-477a-80d5-48713fd96670 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:35.772307 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66084e-1faa-477a-80d5-48713fd96670 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:35.775475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 272/544] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:35.784889 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c815-58df-4ce4-99d7-4c9c97a94432 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:35.785372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c815-58df-4ce4-99d7-4c9c97a94432 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:35.785808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9c815-58df-4ce4-99d7-4c9c97a94432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:35.792863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66084e-1faa-477a-80d5-48713fd96670 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:35.850422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-61e9c815-58df-4ce4-99d7-4c9c97a94432 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:35.853658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-61e9c815-58df-4ce4-99d7-4c9c97a94432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:36.035245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 273/545] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:36.134723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66084e-1faa-477a-80d5-48713fd96670 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:36.232074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8352530-462a-464e-a70b-4ae2eeb848e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:36.233718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8352530-462a-464e-a70b-4ae2eeb848e7 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:36.234484 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8352530-462a-464e-a70b-4ae2eeb848e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:36.235268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8352530-462a-464e-a70b-4ae2eeb848e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:36.259919 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8352530-462a-464e-a70b-4ae2eeb848e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-f8352530-462a-464e-a70b-4ae2eeb848e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:36.279032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:36.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:36.282242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:36.282242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:36.282242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:36.282242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:36.282242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:36.282242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:36.282242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:36.282242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:36.282242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:36.282242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:36.282242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:36.282242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:36.287089 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 273/546] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 495 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:36.297498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9c815-58df-4ce4-99d7-4c9c97a94432 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:36.353697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:36.514159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9c815-58df-4ce4-99d7-4c9c97a94432 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:36.604420 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482ceaf-a82d-48b4-a614-a5918114fd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:36.605431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482ceaf-a82d-48b4-a614-a5918114fd7c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:36.605923 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482ceaf-a82d-48b4-a614-a5918114fd7c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:36.606358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482ceaf-a82d-48b4-a614-a5918114fd7c None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:36.607568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482ceaf-a82d-48b4-a614-a5918114fd7c None admin] RBAC: Adding query filter params (name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:36.608096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482ceaf-a82d-48b4-a614-a5918114fd7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:36.610463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482ceaf-a82d-48b4-a614-a5918114fd7c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:36.633221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 274/547] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:36.643017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8352530-462a-464e-a70b-4ae2eeb848e7 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:36.709461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:36.793257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482ceaf-a82d-48b4-a614-a5918114fd7c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:36.857274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8352530-462a-464e-a70b-4ae2eeb848e7 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:36.946407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421bb4d-4c6d-4411-b33a-af341fa99df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:36.947579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421bb4d-4c6d-4411-b33a-af341fa99df4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:36.948060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421bb4d-4c6d-4411-b33a-af341fa99df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:36.948534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421bb4d-4c6d-4411-b33a-af341fa99df4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:36.973982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:36.986898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421bb4d-4c6d-4411-b33a-af341fa99df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:36.989560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421bb4d-4c6d-4411-b33a-af341fa99df4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:36.993957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482ceaf-a82d-48b4-a614-a5918114fd7c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:37.042974 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1364-078a-412e-8326-508dd38c65b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:37.044017 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1364-078a-412e-8326-508dd38c65b6 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:37.044155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1364-078a-412e-8326-508dd38c65b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:37.044520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a1364-078a-412e-8326-508dd38c65b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-8421bb4d-4c6d-4411-b33a-af341fa99df4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:37.046694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:37.048408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:37.050829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:37.050829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:37.050829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:37.050829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 274/548] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:37.057335 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a1364-078a-412e-8326-508dd38c65b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:37.061635 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421bb4d-4c6d-4411-b33a-af341fa99df4 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-440a1364-078a-412e-8326-508dd38c65b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:37.063277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:37.066612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:37.069758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:37.069758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:37.069758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:37.069758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:37.069758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:37.069758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:37.069758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:37.069758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:37.069758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:37.069758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:37.069758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:37.069758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:37.069758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 275/549] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:37.087140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1364-078a-412e-8326-508dd38c65b6 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:37.222117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:37.280416 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421bb4d-4c6d-4411-b33a-af341fa99df4 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:37.376582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027166d-3b4c-4a88-bcb6-42e71f445af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:37.377800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027166d-3b4c-4a88-bcb6-42e71f445af1 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:37.378276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027166d-3b4c-4a88-bcb6-42e71f445af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:37.378753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027166d-3b4c-4a88-bcb6-42e71f445af1 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:37.383145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027166d-3b4c-4a88-bcb6-42e71f445af1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:37.383145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027166d-3b4c-4a88-bcb6-42e71f445af1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:37.383658 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027166d-3b4c-4a88-bcb6-42e71f445af1 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:37.410865 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 275/550] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:37.423417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027166d-3b4c-4a88-bcb6-42e71f445af1 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:37.425991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a1364-078a-412e-8326-508dd38c65b6 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:37.547813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ffef1-29b3-4c82-bd3f-d8eee9877843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:37.548843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ffef1-29b3-4c82-bd3f-d8eee9877843 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:37.549347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ffef1-29b3-4c82-bd3f-d8eee9877843 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:37.549830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ffef1-29b3-4c82-bd3f-d8eee9877843 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:37.551128 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ffef1-29b3-4c82-bd3f-d8eee9877843 None admin] RBAC: Adding query filter params (name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:37.551685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ffef1-29b3-4c82-bd3f-d8eee9877843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:37.554431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ffef1-29b3-4c82-bd3f-d8eee9877843 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:37.580489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 276/551] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 499 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:37.590225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ffef1-29b3-4c82-bd3f-d8eee9877843 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:37.620208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:37.663486 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027166d-3b4c-4a88-bcb6-42e71f445af1 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:37.788690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684676d-5195-4be2-9690-396f27688ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:37.788690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684676d-5195-4be2-9690-396f27688ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:37.789151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684676d-5195-4be2-9690-396f27688ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:37.789151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684676d-5195-4be2-9690-396f27688ea4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:37.795704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:37.812232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684676d-5195-4be2-9690-396f27688ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:37.813703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684676d-5195-4be2-9690-396f27688ea4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-4684676d-5195-4be2-9690-396f27688ea4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:37.847956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:37.850003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:37.852065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:37.852065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:37.852065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:37.852065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:37.852065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:37.852065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:37.852065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:37.852065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:37.852065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:00:37.852065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:37.852065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 276/552] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:00:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:37.864223 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684676d-5195-4be2-9690-396f27688ea4 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:37.866869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ffef1-29b3-4c82-bd3f-d8eee9877843 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:37.949240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0539d5-da1a-41ea-b724-fa4d6210403e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:37.950561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0539d5-da1a-41ea-b724-fa4d6210403e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:37.951063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0539d5-da1a-41ea-b724-fa4d6210403e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:37.951594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0539d5-da1a-41ea-b724-fa4d6210403e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:37.975658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0539d5-da1a-41ea-b724-fa4d6210403e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:37.981128 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0539d5-da1a-41ea-b724-fa4d6210403e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-0b0539d5-da1a-41ea-b724-fa4d6210403e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:38.015307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:38.017182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:38.018903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:38.018903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:38.018903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:38.018903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 277/553] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:38.029273 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0539d5-da1a-41ea-b724-fa4d6210403e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:38.236136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:38.265785 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684676d-5195-4be2-9690-396f27688ea4 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:38.370742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d134b-75a5-449c-9f09-31975041c42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:38.372726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d134b-75a5-449c-9f09-31975041c42d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:38.373226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d134b-75a5-449c-9f09-31975041c42d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:38.374078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d134b-75a5-449c-9f09-31975041c42d None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:38.377186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d134b-75a5-449c-9f09-31975041c42d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:38.377960 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d134b-75a5-449c-9f09-31975041c42d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:38.381840 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d134b-75a5-449c-9f09-31975041c42d None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:38.382097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0539d5-da1a-41ea-b724-fa4d6210403e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:38.426857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 277/554] 10.222.0.61 () {56 vars in 1135 bytes} [Wed Sep 7 13:00:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 563 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:38.428210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d0727-3b52-4179-9516-eee5f47bf602 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:38.428549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d0727-3b52-4179-9516-eee5f47bf602 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:38.428800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d0727-3b52-4179-9516-eee5f47bf602 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:38.430389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 278/555] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:38.443060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51cae3-708c-4c6e-a920-f7adc60f6374 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:38.443504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51cae3-708c-4c6e-a920-f7adc60f6374 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:38.447134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51cae3-708c-4c6e-a920-f7adc60f6374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:38.447686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c14224-12d7-46de-a687-b517663fa0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:38.447686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c14224-12d7-46de-a687-b517663fa0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:38.448022 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c14224-12d7-46de-a687-b517663fa0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:38.448487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c14224-12d7-46de-a687-b517663fa0b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:38.449682 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c14224-12d7-46de-a687-b517663fa0b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:38.450289 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c14224-12d7-46de-a687-b517663fa0b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:38.455208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c14224-12d7-46de-a687-b517663fa0b2 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:38.465695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-ec51cae3-708c-4c6e-a920-f7adc60f6374 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:38.467042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ec51cae3-708c-4c6e-a920-f7adc60f6374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:38.486385 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 278/556] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:00:38.487447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c14224-12d7-46de-a687-b517663fa0b2 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:38.545153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:38.591540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 279/557] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:38.593368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51cae3-708c-4c6e-a920-f7adc60f6374 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:38.731809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c14224-12d7-46de-a687-b517663fa0b2 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:38.780011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:38.819711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8c4e3-b2e7-49dd-9c78-cd724d971e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:38.820813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8c4e3-b2e7-49dd-9c78-cd724d971e64 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:38.823921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8c4e3-b2e7-49dd-9c78-cd724d971e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:38.824960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8c4e3-b2e7-49dd-9c78-cd724d971e64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:38.855553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51cae3-708c-4c6e-a920-f7adc60f6374 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:38.925909 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b7483-4680-4fc9-a8e6-e03047b9bb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:38.927236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b7483-4680-4fc9-a8e6-e03047b9bb30 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:38.927757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b7483-4680-4fc9-a8e6-e03047b9bb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:38.928264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b7483-4680-4fc9-a8e6-e03047b9bb30 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:38.940305 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b7483-4680-4fc9-a8e6-e03047b9bb30 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:38.942282 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b7483-4680-4fc9-a8e6-e03047b9bb30 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-5c4b7483-4680-4fc9-a8e6-e03047b9bb30 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:38.958319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:38.960970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:38.972077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:38.972077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:38.972077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:38.972077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:38.972077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:38.972077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:38.972077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:38.972077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:38.972077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:00:38.972077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:38.972077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 280/558] 10.222.0.61 () {56 vars in 1093 bytes} [Wed Sep 7 13:00:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:38.972077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37e0d0-dc9b-4083-8bb4-e27ad8d945a2 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:38.972077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37e0d0-dc9b-4083-8bb4-e27ad8d945a2 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:38.973794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37e0d0-dc9b-4083-8bb4-e27ad8d945a2 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:38.973794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 281/559] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:38.973794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37e0d0-dc9b-4083-8bb4-e27ad8d945a2 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:39.005085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8c4e3-b2e7-49dd-9c78-cd724d971e64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:39.010856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8c4e3-b2e7-49dd-9c78-cd724d971e64 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:39.078112 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:39.196079 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 279/560] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 709 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:39.197463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37e0d0-dc9b-4083-8bb4-e27ad8d945a2 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:39.198475 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8c4e3-b2e7-49dd-9c78-cd724d971e64 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:39.239919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:39.267164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a239ec-65e3-4377-9d62-8137d3893838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:39.269108 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a239ec-65e3-4377-9d62-8137d3893838 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:39.269822 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a239ec-65e3-4377-9d62-8137d3893838 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:39.270322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a239ec-65e3-4377-9d62-8137d3893838 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:39.291627 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a239ec-65e3-4377-9d62-8137d3893838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-95a239ec-65e3-4377-9d62-8137d3893838 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:39.302065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:39.303124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:39.303979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:39.303979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:39.303979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:39.303979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:39.303979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:39.303979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:39.303979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:39.303979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:39.303979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:39.303979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:39.303979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:39.303979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:39.304687 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 282/561] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:39.310310 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896e511-c57c-4516-9540-2ce4c0975df0 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:39.310751 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896e511-c57c-4516-9540-2ce4c0975df0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:39.311281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896e511-c57c-4516-9540-2ce4c0975df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:39.340880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-d896e511-c57c-4516-9540-2ce4c0975df0 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:39.342470 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d896e511-c57c-4516-9540-2ce4c0975df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:39.370538 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8c4e3-b2e7-49dd-9c78-cd724d971e64 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:39.425354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e596f-629a-4cae-8424-8af0641a8294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:39.426381 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e596f-629a-4cae-8424-8af0641a8294 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:39.426850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e596f-629a-4cae-8424-8af0641a8294 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:39.427342 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e596f-629a-4cae-8424-8af0641a8294 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:39.428107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 283/562] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:39.428703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e596f-629a-4cae-8424-8af0641a8294 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:39.429171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896e511-c57c-4516-9540-2ce4c0975df0 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:39.429389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e596f-629a-4cae-8424-8af0641a8294 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:39.432271 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e596f-629a-4cae-8424-8af0641a8294 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:39.475630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 280/563] 10.222.0.61 () {56 vars in 1133 bytes} [Wed Sep 7 13:00:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:39.475630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e596f-629a-4cae-8424-8af0641a8294 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:39.478745 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:39.591064 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:39.605196 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896e511-c57c-4516-9540-2ce4c0975df0 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:39.628004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e596f-629a-4cae-8424-8af0641a8294 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:39.670671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf472393-c566-4cb9-9d00-9c758cf4ff6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:39.671574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf472393-c566-4cb9-9d00-9c758cf4ff6f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:39.672051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf472393-c566-4cb9-9d00-9c758cf4ff6f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:39.672520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf472393-c566-4cb9-9d00-9c758cf4ff6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:39.691088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf472393-c566-4cb9-9d00-9c758cf4ff6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:39.693793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf472393-c566-4cb9-9d00-9c758cf4ff6f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:39.695327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12496ff8-a531-46a2-a271-c428a5ae7fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:39.697304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12496ff8-a531-46a2-a271-c428a5ae7fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:39.697858 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12496ff8-a531-46a2-a271-c428a5ae7fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:39.698305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12496ff8-a531-46a2-a271-c428a5ae7fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:39.699489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12496ff8-a531-46a2-a271-c428a5ae7fa3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:39.699999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12496ff8-a531-46a2-a271-c428a5ae7fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:39.702423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12496ff8-a531-46a2-a271-c428a5ae7fa3 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:39.722671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 281/564] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:39.722671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f779ca0-fda5-45e0-a3c5-18bf647cdd26 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:39.722671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f779ca0-fda5-45e0-a3c5-18bf647cdd26 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:39.722671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f779ca0-fda5-45e0-a3c5-18bf647cdd26 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-bf472393-c566-4cb9-9d00-9c758cf4ff6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:39.723301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:39.726972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:39.729243 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:39.729243 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:39.729243 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:39.729601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 282/565] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:39.729601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f779ca0-fda5-45e0-a3c5-18bf647cdd26 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:39.729843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 284/566] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:39.729843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf472393-c566-4cb9-9d00-9c758cf4ff6f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:39.869879 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:39.887361 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf472393-c566-4cb9-9d00-9c758cf4ff6f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:39.888179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f779ca0-fda5-45e0-a3c5-18bf647cdd26 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:39.939662 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc353dc-cbf9-44be-bf6c-0f344e13b696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:39.939989 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ef21e-cbdd-4d48-9c53-982ba3c39428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:39.940451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef21e-cbdd-4d48-9c53-982ba3c39428 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:39.940622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc353dc-cbf9-44be-bf6c-0f344e13b696 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:39.940886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef21e-cbdd-4d48-9c53-982ba3c39428 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:39.941061 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc353dc-cbf9-44be-bf6c-0f344e13b696 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:39.941315 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef21e-cbdd-4d48-9c53-982ba3c39428 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:39.941470 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc353dc-cbf9-44be-bf6c-0f344e13b696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:39.954508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:39.956752 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ef21e-cbdd-4d48-9c53-982ba3c39428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:39.958773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc353dc-cbf9-44be-bf6c-0f344e13b696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:39.961890 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc353dc-cbf9-44be-bf6c-0f344e13b696 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-bc3ef21e-cbdd-4d48-9c53-982ba3c39428 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:39.968663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:39.970549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:39.972319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:39.972319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:39.972319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:39.972319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:39.972319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:39.972319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:39.972319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:39.972319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:39.972319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:39.972319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:39.972319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:39.972319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:39.972319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 285/567] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:39.973674 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb085ea-fecb-4a36-a2be-0363aafd1d81 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:39.973674 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb085ea-fecb-4a36-a2be-0363aafd1d81 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:39.975409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb085ea-fecb-4a36-a2be-0363aafd1d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-bcc353dc-cbf9-44be-bf6c-0f344e13b696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:39.989358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:39.991281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:39.993267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:39.993267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:39.993267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:39.993267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 283/568] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:39.993267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc353dc-cbf9-44be-bf6c-0f344e13b696 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:40.008776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-cdb085ea-fecb-4a36-a2be-0363aafd1d81 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:40.010287 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-cdb085ea-fecb-4a36-a2be-0363aafd1d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:40.113129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 286/569] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:40.114726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb085ea-fecb-4a36-a2be-0363aafd1d81 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:40.135628 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc353dc-cbf9-44be-bf6c-0f344e13b696 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:40.182429 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3ad45-df22-4c8e-8af5-44819c914d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:40.183315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3ad45-df22-4c8e-8af5-44819c914d6c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:40.183688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3ad45-df22-4c8e-8af5-44819c914d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:40.184056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3ad45-df22-4c8e-8af5-44819c914d6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:40.197175 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3ad45-df22-4c8e-8af5-44819c914d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:40.198853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3ad45-df22-4c8e-8af5-44819c914d6c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-9bb3ad45-df22-4c8e-8af5-44819c914d6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:40.216540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:40.218619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:40.220670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:40.220670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:40.220670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:40.220670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 284/570] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:40.224321 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3ad45-df22-4c8e-8af5-44819c914d6c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:40.274000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:40.299942 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:40.308490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb085ea-fecb-4a36-a2be-0363aafd1d81 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:40.368465 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f4f84-ea77-4e97-b4e9-f54acd51bd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:40.369390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f4f84-ea77-4e97-b4e9-f54acd51bd02 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:40.369672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f4f84-ea77-4e97-b4e9-f54acd51bd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:40.369956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f4f84-ea77-4e97-b4e9-f54acd51bd02 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:40.370635 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f4f84-ea77-4e97-b4e9-f54acd51bd02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:40.371699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f4f84-ea77-4e97-b4e9-f54acd51bd02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:40.373210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f4f84-ea77-4e97-b4e9-f54acd51bd02 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:40.395815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 287/571] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:40.395815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f4f84-ea77-4e97-b4e9-f54acd51bd02 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:40.402985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3ad45-df22-4c8e-8af5-44819c914d6c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:40.466625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4cb43-b782-495c-b858-0cd11291453c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:40.467533 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4cb43-b782-495c-b858-0cd11291453c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:40.467876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4cb43-b782-495c-b858-0cd11291453c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:40.468224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4cb43-b782-495c-b858-0cd11291453c None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:40.468998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4cb43-b782-495c-b858-0cd11291453c None admin] RBAC: Adding query filter params (name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:40.469363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4cb43-b782-495c-b858-0cd11291453c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:40.470966 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4cb43-b782-495c-b858-0cd11291453c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:40.481482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 285/572] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:40.482582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4cb43-b782-495c-b858-0cd11291453c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:40.560974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f4f84-ea77-4e97-b4e9-f54acd51bd02 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:40.615509 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4cb43-b782-495c-b858-0cd11291453c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:40.621249 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b0016-9936-4ca6-a847-c7709b3ad74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:40.622283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b0016-9936-4ca6-a847-c7709b3ad74d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:40.622768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b0016-9936-4ca6-a847-c7709b3ad74d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:40.623264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b0016-9936-4ca6-a847-c7709b3ad74d None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:40.623944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b0016-9936-4ca6-a847-c7709b3ad74d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:40.624397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b0016-9936-4ca6-a847-c7709b3ad74d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:40.624564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:40.626200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b0016-9936-4ca6-a847-c7709b3ad74d None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:40.629442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:40.641124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 288/573] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:40.642654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b0016-9936-4ca6-a847-c7709b3ad74d None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:40.660256 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a30f0a-b16a-4def-91dc-0c4704367708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:40.660898 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a30f0a-b16a-4def-91dc-0c4704367708 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:40.661247 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a30f0a-b16a-4def-91dc-0c4704367708 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:40.661613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a30f0a-b16a-4def-91dc-0c4704367708 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:40.662282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a30f0a-b16a-4def-91dc-0c4704367708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:40.662585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a30f0a-b16a-4def-91dc-0c4704367708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:40.664047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a30f0a-b16a-4def-91dc-0c4704367708 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:40.678805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 286/574] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:40.679919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a30f0a-b16a-4def-91dc-0c4704367708 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:40.783544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b0016-9936-4ca6-a847-c7709b3ad74d None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:40.823176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7863e-c4a7-484f-8de6-901c7e0edc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:40.823841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7863e-c4a7-484f-8de6-901c7e0edc57 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:40.824159 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7863e-c4a7-484f-8de6-901c7e0edc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:40.824310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a30f0a-b16a-4def-91dc-0c4704367708 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:40.824450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7863e-c4a7-484f-8de6-901c7e0edc57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:40.833675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7863e-c4a7-484f-8de6-901c7e0edc57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:40.835218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7863e-c4a7-484f-8de6-901c7e0edc57 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-59b7863e-c4a7-484f-8de6-901c7e0edc57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:40.849949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:40.854045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:40.855559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:40.855559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:40.855559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:40.855559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:40.855559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:40.855559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:40.855559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:40.855559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:40.856954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 289/575] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:40.858033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7863e-c4a7-484f-8de6-901c7e0edc57 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:40.875467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2820c31e-ac59-435f-b262-8431b7eaa178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:40.876125 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820c31e-ac59-435f-b262-8431b7eaa178 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:40.876403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820c31e-ac59-435f-b262-8431b7eaa178 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:40.876938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820c31e-ac59-435f-b262-8431b7eaa178 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:40.890427 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2820c31e-ac59-435f-b262-8431b7eaa178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-2820c31e-ac59-435f-b262-8431b7eaa178 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:40.895650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:40.897928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 287/576] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:40.900476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2820c31e-ac59-435f-b262-8431b7eaa178 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:40.932712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:40.960023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:41.012383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7863e-c4a7-484f-8de6-901c7e0edc57 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:41.052403 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96fc87-fba0-489d-ba42-e80736aa2816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:41.052907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2820c31e-ac59-435f-b262-8431b7eaa178 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:41.053254 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96fc87-fba0-489d-ba42-e80736aa2816 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:41.053617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96fc87-fba0-489d-ba42-e80736aa2816 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:41.053985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96fc87-fba0-489d-ba42-e80736aa2816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:41.069413 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96fc87-fba0-489d-ba42-e80736aa2816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:41.071034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96fc87-fba0-489d-ba42-e80736aa2816 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-3b96fc87-fba0-489d-ba42-e80736aa2816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:41.097466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:41.099276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:41.101613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:41.101613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:41.101613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:41.101613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 290/577] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:41.101613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96fc87-fba0-489d-ba42-e80736aa2816 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:41.115455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db287b90-999f-47ae-b2d4-ad733a027847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:41.116557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db287b90-999f-47ae-b2d4-ad733a027847 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:41.116894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db287b90-999f-47ae-b2d4-ad733a027847 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:41.117223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db287b90-999f-47ae-b2d4-ad733a027847 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:41.127180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db287b90-999f-47ae-b2d4-ad733a027847 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:41.129129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db287b90-999f-47ae-b2d4-ad733a027847 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-db287b90-999f-47ae-b2d4-ad733a027847 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:41.144158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:41.154801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:41.156879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:41.156879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:41.156879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:41.156879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:41.156879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:41.156879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:41.156879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:41.156879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:41.156879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 288/578] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:41.156879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db287b90-999f-47ae-b2d4-ad733a027847 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:41.242715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96fc87-fba0-489d-ba42-e80736aa2816 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:41.257018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:41.261405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:41.280575 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db287b90-999f-47ae-b2d4-ad733a027847 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:41.298586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef3dde-1bf3-41a4-8856-c9d685744f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:41.299450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef3dde-1bf3-41a4-8856-c9d685744f1e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:41.299641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef3dde-1bf3-41a4-8856-c9d685744f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:41.299793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef3dde-1bf3-41a4-8856-c9d685744f1e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:41.313544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef3dde-1bf3-41a4-8856-c9d685744f1e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:41.316400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef3dde-1bf3-41a4-8856-c9d685744f1e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:41.328241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73806e9a-4e7c-4ede-a139-5a697fbe41eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:41.328909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73806e9a-4e7c-4ede-a139-5a697fbe41eb None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:41.329122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73806e9a-4e7c-4ede-a139-5a697fbe41eb None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:41.329384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73806e9a-4e7c-4ede-a139-5a697fbe41eb None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:41.330095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73806e9a-4e7c-4ede-a139-5a697fbe41eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:41.330396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73806e9a-4e7c-4ede-a139-5a697fbe41eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:41.331967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73806e9a-4e7c-4ede-a139-5a697fbe41eb None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-67ef3dde-1bf3-41a4-8856-c9d685744f1e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:41.340723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:41.342812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:41.344009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:41.344009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:41.344009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:41.344009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:41.344009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:41.344009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:41.344009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:41.344009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:41.344009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:00:41.344009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:41.344009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 291/579] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:41.344928 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef3dde-1bf3-41a4-8856-c9d685744f1e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:41.353249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 289/580] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:41.355087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73806e9a-4e7c-4ede-a139-5a697fbe41eb None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:41.480961 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73806e9a-4e7c-4ede-a139-5a697fbe41eb None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:41.498771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef3dde-1bf3-41a4-8856-c9d685744f1e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:41.526834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bf004-c5b8-4604-b5eb-b52f0e5041e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:41.527686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bf004-c5b8-4604-b5eb-b52f0e5041e8 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:41.528036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bf004-c5b8-4604-b5eb-b52f0e5041e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:41.528433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bf004-c5b8-4604-b5eb-b52f0e5041e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:41.529237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bf004-c5b8-4604-b5eb-b52f0e5041e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:41.529657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bf004-c5b8-4604-b5eb-b52f0e5041e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:41.531419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bf004-c5b8-4604-b5eb-b52f0e5041e8 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:41.541041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82859f1-5ad9-47cc-93fa-835323e69720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:41.541666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82859f1-5ad9-47cc-93fa-835323e69720 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:41.541944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82859f1-5ad9-47cc-93fa-835323e69720 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:41.542215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82859f1-5ad9-47cc-93fa-835323e69720 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:41.542887 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82859f1-5ad9-47cc-93fa-835323e69720 None admin] RBAC: Adding query filter params (name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:41.543225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82859f1-5ad9-47cc-93fa-835323e69720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:41.544627 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82859f1-5ad9-47cc-93fa-835323e69720 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:41.549547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 290/581] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:41.550760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bf004-c5b8-4604-b5eb-b52f0e5041e8 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:41.555059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 292/582] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:41.556252 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82859f1-5ad9-47cc-93fa-835323e69720 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:41.565084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:41.596642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:41.690606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82859f1-5ad9-47cc-93fa-835323e69720 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:41.713053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bf004-c5b8-4604-b5eb-b52f0e5041e8 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:41.749386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44c20f-62d0-4b62-bf36-7516eb41fa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:41.749933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44c20f-62d0-4b62-bf36-7516eb41fa4d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:41.750311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44c20f-62d0-4b62-bf36-7516eb41fa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:41.750593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44c20f-62d0-4b62-bf36-7516eb41fa4d None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:41.751404 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44c20f-62d0-4b62-bf36-7516eb41fa4d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:41.751964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44c20f-62d0-4b62-bf36-7516eb41fa4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:41.753938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44c20f-62d0-4b62-bf36-7516eb41fa4d None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:41.756865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab5b25-18ce-4bd3-b6fc-b0bb9ff97d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:41.757816 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab5b25-18ce-4bd3-b6fc-b0bb9ff97d4f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:41.758228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab5b25-18ce-4bd3-b6fc-b0bb9ff97d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:41.758596 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab5b25-18ce-4bd3-b6fc-b0bb9ff97d4f None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:41.760235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab5b25-18ce-4bd3-b6fc-b0bb9ff97d4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:41.760772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab5b25-18ce-4bd3-b6fc-b0bb9ff97d4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:41.762676 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab5b25-18ce-4bd3-b6fc-b0bb9ff97d4f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:41.775954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 293/583] 10.222.0.61 () {56 vars in 1137 bytes} [Wed Sep 7 13:00:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:41.777231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44c20f-62d0-4b62-bf36-7516eb41fa4d None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:41.777921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 291/584] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:41.779294 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab5b25-18ce-4bd3-b6fc-b0bb9ff97d4f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:41.868964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:41.904080 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:41.918366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44c20f-62d0-4b62-bf36-7516eb41fa4d None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:41.935890 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab5b25-18ce-4bd3-b6fc-b0bb9ff97d4f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:41.957177 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afe9eb-7466-4cfc-9d31-0b68c1251074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:41.957803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afe9eb-7466-4cfc-9d31-0b68c1251074 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:41.958097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afe9eb-7466-4cfc-9d31-0b68c1251074 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:41.958364 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afe9eb-7466-4cfc-9d31-0b68c1251074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:41.959238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afe9eb-7466-4cfc-9d31-0b68c1251074 None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=0f886e0999d34bf9870b448772cc38a3) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:41.959560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afe9eb-7466-4cfc-9d31-0b68c1251074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:41.961053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afe9eb-7466-4cfc-9d31-0b68c1251074 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:41.968378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 294/585] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:00:41 2022] GET /identity/v3/role_assignments?user.id=0f886e0999d34bf9870b448772cc38a3&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:41.969422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afe9eb-7466-4cfc-9d31-0b68c1251074 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:41.999211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f93f03-4007-4724-8dcb-5e58cefab299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:42.000364 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f93f03-4007-4724-8dcb-5e58cefab299 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:42.000762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f93f03-4007-4724-8dcb-5e58cefab299 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:42.001245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f93f03-4007-4724-8dcb-5e58cefab299 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:42.017644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f93f03-4007-4724-8dcb-5e58cefab299 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:42.021089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f93f03-4007-4724-8dcb-5e58cefab299 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-a3f93f03-4007-4724-8dcb-5e58cefab299 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:42.043689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:42.045607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:42.047537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:42.047537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:42.047537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:42.047537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:42.047537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:42.047537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:42.047537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:42.047537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:42.047537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 13:00:42.047537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:42.048600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 292/586] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:42.050468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f93f03-4007-4724-8dcb-5e58cefab299 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:42.179147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afe9eb-7466-4cfc-9d31-0b68c1251074 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:42.226260 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b0689-6825-4477-96de-6f6854aa14dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:42.227047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b0689-6825-4477-96de-6f6854aa14dd None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:42.227364 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b0689-6825-4477-96de-6f6854aa14dd None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:42.227672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b0689-6825-4477-96de-6f6854aa14dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:42.242231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b0689-6825-4477-96de-6f6854aa14dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:42.244362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b0689-6825-4477-96de-6f6854aa14dd None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-275b0689-6825-4477-96de-6f6854aa14dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:42.266407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:42.268543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:42.270686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:42.270686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:42.270686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:42.270686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 295/587] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:42.270686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b0689-6825-4477-96de-6f6854aa14dd None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:42.274590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:42.282713 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f93f03-4007-4724-8dcb-5e58cefab299 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:42.338986 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:42.353503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e566b-d74c-4845-930e-9c2f35ea0321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:42.354277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e566b-d74c-4845-930e-9c2f35ea0321 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:42.354623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e566b-d74c-4845-930e-9c2f35ea0321 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:42.354957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e566b-d74c-4845-930e-9c2f35ea0321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:42.372547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e566b-d74c-4845-930e-9c2f35ea0321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:42.375952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e566b-d74c-4845-930e-9c2f35ea0321 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-c48e566b-d74c-4845-930e-9c2f35ea0321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:42.405632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:42.407992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:42.409907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:42.409907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:42.409907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:42.409907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 293/588] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:42.409907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e566b-d74c-4845-930e-9c2f35ea0321 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:42.450342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b0689-6825-4477-96de-6f6854aa14dd None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:42.523159 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51df675-e464-4373-aa42-e14dccb2277f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:42.524207 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51df675-e464-4373-aa42-e14dccb2277f None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:42.525236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51df675-e464-4373-aa42-e14dccb2277f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:42.525729 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51df675-e464-4373-aa42-e14dccb2277f None admin] PATH_INFO: `/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/ba93e2d4aaed4c5595d5fd7363626bcb/roles/5f8e244e8257481e85e65a504aa4bb74` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:42.553735 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51df675-e464-4373-aa42-e14dccb2277f None admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user_id=ba93e2d4aaed4c5595d5fd7363626bcb, role_id=5f8e244e8257481e85e65a504aa4bb74)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:42.557624 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51df675-e464-4373-aa42-e14dccb2277f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:42.579055 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e566b-d74c-4845-930e-9c2f35ea0321 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:42.581361 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 296/589] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:00:42 2022] PUT /identity/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/ba93e2d4aaed4c5595d5fd7363626bcb/roles/5f8e244e8257481e85e65a504aa4bb74 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:00:42.582517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51df675-e464-4373-aa42-e14dccb2277f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:42.653530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f480f-0af4-48dd-a199-787b13b0d6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:42.655293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f480f-0af4-48dd-a199-787b13b0d6ca None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:42.655667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f480f-0af4-48dd-a199-787b13b0d6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:42.655907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f480f-0af4-48dd-a199-787b13b0d6ca None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:42.656959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f480f-0af4-48dd-a199-787b13b0d6ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:42.657271 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f480f-0af4-48dd-a199-787b13b0d6ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:42.658900 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f480f-0af4-48dd-a199-787b13b0d6ca None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:42.687704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:42.698389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 294/590] 10.222.0.61 () {56 vars in 1137 bytes} [Wed Sep 7 13:00:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:42.699393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f480f-0af4-48dd-a199-787b13b0d6ca None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:42.766736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:42.778949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51df675-e464-4373-aa42-e14dccb2277f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:42.828611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cd7fa-5012-44e9-89d9-dd31d394b542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:42.829654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cd7fa-5012-44e9-89d9-dd31d394b542 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:42.830155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cd7fa-5012-44e9-89d9-dd31d394b542 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:42.830625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cd7fa-5012-44e9-89d9-dd31d394b542 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:42.831896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cd7fa-5012-44e9-89d9-dd31d394b542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:42.832451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cd7fa-5012-44e9-89d9-dd31d394b542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:42.836671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cd7fa-5012-44e9-89d9-dd31d394b542 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:42.866271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 297/591] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:42.868025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cd7fa-5012-44e9-89d9-dd31d394b542 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:42.881696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f480f-0af4-48dd-a199-787b13b0d6ca None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:42.931480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49c200-0257-4b16-88ac-f221bb31a903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:42.933092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49c200-0257-4b16-88ac-f221bb31a903 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:42.933775 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49c200-0257-4b16-88ac-f221bb31a903 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:42.934345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49c200-0257-4b16-88ac-f221bb31a903 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:42.935346 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49c200-0257-4b16-88ac-f221bb31a903 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:42.935885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49c200-0257-4b16-88ac-f221bb31a903 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:42.938721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49c200-0257-4b16-88ac-f221bb31a903 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:42.963369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 295/592] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:42.964890 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49c200-0257-4b16-88ac-f221bb31a903 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:43.035450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cd7fa-5012-44e9-89d9-dd31d394b542 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:43.053065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:43.080322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd63c04-8dd9-4965-9ad6-d950b2a515ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:43.081388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd63c04-8dd9-4965-9ad6-d950b2a515ca None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:43.081861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd63c04-8dd9-4965-9ad6-d950b2a515ca None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:43.082351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd63c04-8dd9-4965-9ad6-d950b2a515ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:43.096006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd63c04-8dd9-4965-9ad6-d950b2a515ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:43.097862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd63c04-8dd9-4965-9ad6-d950b2a515ca None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-ecd63c04-8dd9-4965-9ad6-d950b2a515ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:43.115760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:43.116777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:43.117666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:43.117666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:43.117666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:43.120177 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 298/593] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:43.121637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd63c04-8dd9-4965-9ad6-d950b2a515ca None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:43.172642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:43.181291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49c200-0257-4b16-88ac-f221bb31a903 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:43.230131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827bdfd-acae-4a24-9823-8c8f482ff88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:43.231072 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827bdfd-acae-4a24-9823-8c8f482ff88e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:43.231570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827bdfd-acae-4a24-9823-8c8f482ff88e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:43.231994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827bdfd-acae-4a24-9823-8c8f482ff88e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:43.241427 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827bdfd-acae-4a24-9823-8c8f482ff88e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:43.243765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827bdfd-acae-4a24-9823-8c8f482ff88e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:43.250016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd63c04-8dd9-4965-9ad6-d950b2a515ca None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-b827bdfd-acae-4a24-9823-8c8f482ff88e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:43.258044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:43.259938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:43.260906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:43.260906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:43.260906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:43.260906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:43.260906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:43.260906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:43.260906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:43.260906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:43.260906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 13:00:43.260906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:43.263456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 296/594] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:43.264742 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827bdfd-acae-4a24-9823-8c8f482ff88e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:43.295742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c7fac-ece5-44e5-9550-7bef761d3548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:43.296502 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c7fac-ece5-44e5-9550-7bef761d3548 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:43.296914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c7fac-ece5-44e5-9550-7bef761d3548 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:43.297181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c7fac-ece5-44e5-9550-7bef761d3548 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:43.308224 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c7fac-ece5-44e5-9550-7bef761d3548 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:43.310915 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c7fac-ece5-44e5-9550-7bef761d3548 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-1c0c7fac-ece5-44e5-9550-7bef761d3548 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:43.327824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:43.328834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:43.330160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:43.330160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:43.330160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:43.330160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:43.330160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:43.330160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:43.330160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:43.330160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:43.330160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 299/595] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:43.330160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c7fac-ece5-44e5-9550-7bef761d3548 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:43.372105 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:43.418214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827bdfd-acae-4a24-9823-8c8f482ff88e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:43.466637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c7fac-ece5-44e5-9550-7bef761d3548 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:43.467834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c06fd-6622-40f6-a3b3-6d2fd845f197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:43.468535 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c06fd-6622-40f6-a3b3-6d2fd845f197 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:43.468873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c06fd-6622-40f6-a3b3-6d2fd845f197 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:43.469211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c06fd-6622-40f6-a3b3-6d2fd845f197 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:43.470275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c06fd-6622-40f6-a3b3-6d2fd845f197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:43.470657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c06fd-6622-40f6-a3b3-6d2fd845f197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:43.473368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c06fd-6622-40f6-a3b3-6d2fd845f197 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:43.491046 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 297/596] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:43.492539 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c06fd-6622-40f6-a3b3-6d2fd845f197 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:43.514269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9daa40-61f6-4732-908f-07ca87e8ae2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:43.515399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9daa40-61f6-4732-908f-07ca87e8ae2b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:43.516053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9daa40-61f6-4732-908f-07ca87e8ae2b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:43.516519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9daa40-61f6-4732-908f-07ca87e8ae2b None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:43.517500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9daa40-61f6-4732-908f-07ca87e8ae2b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:43.518071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:43.519918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9daa40-61f6-4732-908f-07ca87e8ae2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:43.522035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9daa40-61f6-4732-908f-07ca87e8ae2b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:43.545427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 300/597] 10.222.0.61 () {56 vars in 1139 bytes} [Wed Sep 7 13:00:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:43.546762 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9daa40-61f6-4732-908f-07ca87e8ae2b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:43.644447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c06fd-6622-40f6-a3b3-6d2fd845f197 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:43.659704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:43.680160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9daa40-61f6-4732-908f-07ca87e8ae2b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:43.689593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d1b0d-b8f8-44ee-bd84-0c05462e6cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:43.690638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1b0d-b8f8-44ee-bd84-0c05462e6cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:43.691197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1b0d-b8f8-44ee-bd84-0c05462e6cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:43.691706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1b0d-b8f8-44ee-bd84-0c05462e6cb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:43.692678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d1b0d-b8f8-44ee-bd84-0c05462e6cb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:43.693324 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d1b0d-b8f8-44ee-bd84-0c05462e6cb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:43.695130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d1b0d-b8f8-44ee-bd84-0c05462e6cb9 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:43.715039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 298/598] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:43.716402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d1b0d-b8f8-44ee-bd84-0c05462e6cb9 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:43.722811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bc194-ebf8-4168-a16c-3c994eada22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:43.723501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bc194-ebf8-4168-a16c-3c994eada22f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:43.723780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bc194-ebf8-4168-a16c-3c994eada22f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:43.724052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bc194-ebf8-4168-a16c-3c994eada22f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:43.732391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bc194-ebf8-4168-a16c-3c994eada22f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:43.734002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bc194-ebf8-4168-a16c-3c994eada22f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-9b8bc194-ebf8-4168-a16c-3c994eada22f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:43.749002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:43.750269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:43.752166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:43.752166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:43.752166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:43.752166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:43.752166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:43.752166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:43.752166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:43.752166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:43.752166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 301/599] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:43.752166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bc194-ebf8-4168-a16c-3c994eada22f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:43.816464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:43.845409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d1b0d-b8f8-44ee-bd84-0c05462e6cb9 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:43.885176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bc194-ebf8-4168-a16c-3c994eada22f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:43.888937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6953a-bac6-47d7-b59e-af13e144bfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:43.889687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6953a-bac6-47d7-b59e-af13e144bfc4 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:43.890067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6953a-bac6-47d7-b59e-af13e144bfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:43.890721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6953a-bac6-47d7-b59e-af13e144bfc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:43.904640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6953a-bac6-47d7-b59e-af13e144bfc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:43.906521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6953a-bac6-47d7-b59e-af13e144bfc4 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-89e6953a-bac6-47d7-b59e-af13e144bfc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:43.923441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:43.924625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:43.925699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:43.925699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:43.925699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:43.927464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 299/600] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:43.928682 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6953a-bac6-47d7-b59e-af13e144bfc4 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:43.931462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27200e90-d3b8-4011-aae8-e78b60a1146a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:43.932415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27200e90-d3b8-4011-aae8-e78b60a1146a None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:43.932956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27200e90-d3b8-4011-aae8-e78b60a1146a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:43.933375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27200e90-d3b8-4011-aae8-e78b60a1146a None admin] PATH_INFO: `/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/8356f60a0f684484abc8f8d6fb6a7be6/roles/5f8e244e8257481e85e65a504aa4bb74` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:43.955994 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27200e90-d3b8-4011-aae8-e78b60a1146a None admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user_id=8356f60a0f684484abc8f8d6fb6a7be6, role_id=5f8e244e8257481e85e65a504aa4bb74)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:43.957956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27200e90-d3b8-4011-aae8-e78b60a1146a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:43.961482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:43.977667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 302/601] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:00:43 2022] PUT /identity/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/8356f60a0f684484abc8f8d6fb6a7be6/roles/5f8e244e8257481e85e65a504aa4bb74 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:00:43.978772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27200e90-d3b8-4011-aae8-e78b60a1146a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:44.086657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6953a-bac6-47d7-b59e-af13e144bfc4 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:44.147701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27200e90-d3b8-4011-aae8-e78b60a1146a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:44.173788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:44.201654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee129d-1665-42e6-96a7-58bb1bd6af38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:44.202295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee129d-1665-42e6-96a7-58bb1bd6af38 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:44.202570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee129d-1665-42e6-96a7-58bb1bd6af38 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:44.202885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee129d-1665-42e6-96a7-58bb1bd6af38 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:44.203770 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee129d-1665-42e6-96a7-58bb1bd6af38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:44.204076 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee129d-1665-42e6-96a7-58bb1bd6af38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:44.205607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee129d-1665-42e6-96a7-58bb1bd6af38 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:44.208778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5278b40-03e4-4f3f-a33b-66b88b994653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:44.209404 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5278b40-03e4-4f3f-a33b-66b88b994653 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:44.209675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5278b40-03e4-4f3f-a33b-66b88b994653 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:44.209932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5278b40-03e4-4f3f-a33b-66b88b994653 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:44.210614 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5278b40-03e4-4f3f-a33b-66b88b994653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:44.210913 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5278b40-03e4-4f3f-a33b-66b88b994653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:44.212473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5278b40-03e4-4f3f-a33b-66b88b994653 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:44.235160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 300/602] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:44.244149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 303/603] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:44.246458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee129d-1665-42e6-96a7-58bb1bd6af38 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:44.248080 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5278b40-03e4-4f3f-a33b-66b88b994653 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:44.365262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:44.449960 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5278b40-03e4-4f3f-a33b-66b88b994653 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:44.466337 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee129d-1665-42e6-96a7-58bb1bd6af38 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:44.547679 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f9068-472d-4fd9-b03c-34823bca55c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:44.548787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f9068-472d-4fd9-b03c-34823bca55c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:44.549273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f9068-472d-4fd9-b03c-34823bca55c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:44.549756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f9068-472d-4fd9-b03c-34823bca55c9 None admin] PATH_INFO: `/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/b4f82adce75b44c6baf85b6b96dd0728/roles/5f8e244e8257481e85e65a504aa4bb74` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:44.584766 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18104907-06f3-456d-80b1-059a5228ac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:44.585856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18104907-06f3-456d-80b1-059a5228ac35 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:44.586338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18104907-06f3-456d-80b1-059a5228ac35 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:44.587355 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18104907-06f3-456d-80b1-059a5228ac35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:44.607686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f9068-472d-4fd9-b03c-34823bca55c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user_id=b4f82adce75b44c6baf85b6b96dd0728, role_id=5f8e244e8257481e85e65a504aa4bb74)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:44.610395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f9068-472d-4fd9-b03c-34823bca55c9 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:44.617755 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18104907-06f3-456d-80b1-059a5228ac35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:44.621236 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18104907-06f3-456d-80b1-059a5228ac35 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:44.647280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 304/604] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:00:44 2022] PUT /identity/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/b4f82adce75b44c6baf85b6b96dd0728/roles/5f8e244e8257481e85e65a504aa4bb74 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-18104907-06f3-456d-80b1-059a5228ac35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:44.648276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:44.650012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:44.651752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:44.651752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:44.651752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:44.651752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:44.651752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:44.651752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:44.651752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:44.651752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:44.655931 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 301/605] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:44.663428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f9068-472d-4fd9-b03c-34823bca55c9 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:44.946446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f9068-472d-4fd9-b03c-34823bca55c9 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:44.957209 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:45.047963 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602a105-848b-4e8e-a0d6-da4f8d1fde10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:45.049579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602a105-848b-4e8e-a0d6-da4f8d1fde10 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:45.050400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602a105-848b-4e8e-a0d6-da4f8d1fde10 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:45.051249 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602a105-848b-4e8e-a0d6-da4f8d1fde10 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:45.052919 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602a105-848b-4e8e-a0d6-da4f8d1fde10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:45.053820 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602a105-848b-4e8e-a0d6-da4f8d1fde10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:45.057622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602a105-848b-4e8e-a0d6-da4f8d1fde10 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:45.114158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 305/606] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:45.124275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18104907-06f3-456d-80b1-059a5228ac35 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:45.195579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:45.268612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18104907-06f3-456d-80b1-059a5228ac35 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:45.322976 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e35513-2b77-45e8-9600-d2326db3abc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:45.323702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e35513-2b77-45e8-9600-d2326db3abc9 None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:45.323985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e35513-2b77-45e8-9600-d2326db3abc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:45.324244 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e35513-2b77-45e8-9600-d2326db3abc9 None admin] PATH_INFO: `/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/de05c08e30dd4fff93f1c97aac9553d2/roles/5f8e244e8257481e85e65a504aa4bb74` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:45.342430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e35513-2b77-45e8-9600-d2326db3abc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user_id=de05c08e30dd4fff93f1c97aac9553d2, role_id=5f8e244e8257481e85e65a504aa4bb74)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:45.344157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e35513-2b77-45e8-9600-d2326db3abc9 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:45.365308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 302/607] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:00:45 2022] PUT /identity/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/de05c08e30dd4fff93f1c97aac9553d2/roles/5f8e244e8257481e85e65a504aa4bb74 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:00:45.444486 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5310f3-0b63-48f6-94bd-759db8daa8fc None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:45.444773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5310f3-0b63-48f6-94bd-759db8daa8fc None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:45.445097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5310f3-0b63-48f6-94bd-759db8daa8fc None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:45.446541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 306/608] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:45.452630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8b823-5cc4-4d91-bda8-5f633dea859b None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:45.452934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8b823-5cc4-4d91-bda8-5f633dea859b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:45.453234 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8b823-5cc4-4d91-bda8-5f633dea859b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:45.474426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-c0e8b823-5cc4-4d91-bda8-5f633dea859b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:45.475827 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c0e8b823-5cc4-4d91-bda8-5f633dea859b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:45.589142 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 303/609] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:00:45.677388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d85b8c-442e-4d3c-b89d-8bbcfff3766a None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:45.678313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d85b8c-442e-4d3c-b89d-8bbcfff3766a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:45.678313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d85b8c-442e-4d3c-b89d-8bbcfff3766a None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:45.679703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 307/610] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:45.685657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ae64e-7690-45a2-89d4-a8b997605299 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:45.685948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ae64e-7690-45a2-89d4-a8b997605299 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:45.686200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ae64e-7690-45a2-89d4-a8b997605299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:45.716433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-7b8ae64e-7690-45a2-89d4-a8b997605299 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:45.717678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7b8ae64e-7690-45a2-89d4-a8b997605299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:45.747514 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:45.810009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 304/611] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:46.153057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d85b8c-442e-4d3c-b89d-8bbcfff3766a None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:46.236183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ae64e-7690-45a2-89d4-a8b997605299 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:46.385735 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d85b8c-442e-4d3c-b89d-8bbcfff3766a None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:46.403628 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ae64e-7690-45a2-89d4-a8b997605299 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:46.447027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17de382-bb7b-4152-973c-a18f35f9e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:46.447725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17de382-bb7b-4152-973c-a18f35f9e801 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:46.447999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17de382-bb7b-4152-973c-a18f35f9e801 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:46.448264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17de382-bb7b-4152-973c-a18f35f9e801 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:46.461165 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17de382-bb7b-4152-973c-a18f35f9e801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-e17de382-bb7b-4152-973c-a18f35f9e801 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:46.467207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:46.468240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:46.469523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:46.469523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:46.469523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:46.469523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:46.469523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:46.469523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:46.469523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:46.469523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:46.469523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:46.469523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:46.469523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:46.469523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:46.469523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 305/612] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:46.475325 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17de382-bb7b-4152-973c-a18f35f9e801 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:46.499899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1931a-7b6e-4323-b5e2-3245814258cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:46.500562 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1931a-7b6e-4323-b5e2-3245814258cb None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:46.500838 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1931a-7b6e-4323-b5e2-3245814258cb None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:46.501113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1931a-7b6e-4323-b5e2-3245814258cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:46.525654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1931a-7b6e-4323-b5e2-3245814258cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:46.527270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-f6e1931a-7b6e-4323-b5e2-3245814258cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:46.531600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:46.532606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:46.533684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:46.533684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:46.533684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:46.533684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:46.533684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:46.533684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:46.533684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:46.533684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:46.533684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:46.533684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:46.533684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:46.533684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:46.533684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 308/613] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:46.549896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1931a-7b6e-4323-b5e2-3245814258cb None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:46.663613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:46.834098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17de382-bb7b-4152-973c-a18f35f9e801 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:46.931565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1931a-7b6e-4323-b5e2-3245814258cb None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:46.971135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a537a-4563-4bb1-8c32-882aeb6e5b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:46.971135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a537a-4563-4bb1-8c32-882aeb6e5b6e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:46.971135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a537a-4563-4bb1-8c32-882aeb6e5b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:46.971135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a537a-4563-4bb1-8c32-882aeb6e5b6e None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:46.971969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a537a-4563-4bb1-8c32-882aeb6e5b6e None admin] RBAC: Adding query filter params (name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:46.971969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a537a-4563-4bb1-8c32-882aeb6e5b6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:46.974403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a537a-4563-4bb1-8c32-882aeb6e5b6e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:47.000351 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 306/614] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 526 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:47.006988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a537a-4563-4bb1-8c32-882aeb6e5b6e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:47.012120 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe744ba3-19ed-4468-b595-6bf2dbb4ef7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:47.012817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe744ba3-19ed-4468-b595-6bf2dbb4ef7a None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:47.013095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe744ba3-19ed-4468-b595-6bf2dbb4ef7a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:47.013362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe744ba3-19ed-4468-b595-6bf2dbb4ef7a None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:47.014136 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe744ba3-19ed-4468-b595-6bf2dbb4ef7a None admin] RBAC: Adding query filter params (name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:47.014492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe744ba3-19ed-4468-b595-6bf2dbb4ef7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:47.016000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe744ba3-19ed-4468-b595-6bf2dbb4ef7a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:47.039801 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 309/615] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 491 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:00:47.048673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe744ba3-19ed-4468-b595-6bf2dbb4ef7a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:47.242033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:47.246556 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:47.302275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe744ba3-19ed-4468-b595-6bf2dbb4ef7a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:47.318512 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a537a-4563-4bb1-8c32-882aeb6e5b6e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:47.387707 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444621b-4da2-46c8-a493-66501ee9e381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:47.388436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444621b-4da2-46c8-a493-66501ee9e381 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:47.388718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444621b-4da2-46c8-a493-66501ee9e381 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:47.388998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444621b-4da2-46c8-a493-66501ee9e381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:47.415290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444621b-4da2-46c8-a493-66501ee9e381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:47.416800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444621b-4da2-46c8-a493-66501ee9e381 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:47.434264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763485e5-3830-4ed3-a0f7-9fd29990729c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:47.434942 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763485e5-3830-4ed3-a0f7-9fd29990729c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:47.435262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763485e5-3830-4ed3-a0f7-9fd29990729c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:47.435536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763485e5-3830-4ed3-a0f7-9fd29990729c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-3444621b-4da2-46c8-a493-66501ee9e381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:47.449591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:47.451392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:47.453131 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:47.453131 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:47.453131 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:47.455607 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 310/616] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:47.463741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444621b-4da2-46c8-a493-66501ee9e381 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:47.465296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763485e5-3830-4ed3-a0f7-9fd29990729c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:47.466829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763485e5-3830-4ed3-a0f7-9fd29990729c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-763485e5-3830-4ed3-a0f7-9fd29990729c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:47.510058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:47.514063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:47.517463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:47.517463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:47.517463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:47.518013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 307/617] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 511 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:47.523154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763485e5-3830-4ed3-a0f7-9fd29990729c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:47.703365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444621b-4da2-46c8-a493-66501ee9e381 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:47.780488 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763485e5-3830-4ed3-a0f7-9fd29990729c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:47.797173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12069b8-95b6-437c-839a-9dad405bca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:47.798173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12069b8-95b6-437c-839a-9dad405bca2b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:47.798447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12069b8-95b6-437c-839a-9dad405bca2b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:47.798782 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12069b8-95b6-437c-839a-9dad405bca2b None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:47.799568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12069b8-95b6-437c-839a-9dad405bca2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:47.799884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12069b8-95b6-437c-839a-9dad405bca2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:47.801467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12069b8-95b6-437c-839a-9dad405bca2b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:47.818114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:47.833829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 311/618] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:47.840380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12069b8-95b6-437c-839a-9dad405bca2b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:47.852347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:47.869248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871183ee-2097-4ec2-b6b5-451c9e2e1cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:47.869892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871183ee-2097-4ec2-b6b5-451c9e2e1cff None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:47.870728 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871183ee-2097-4ec2-b6b5-451c9e2e1cff None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:47.871021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871183ee-2097-4ec2-b6b5-451c9e2e1cff None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:47.871750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871183ee-2097-4ec2-b6b5-451c9e2e1cff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:47.872389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871183ee-2097-4ec2-b6b5-451c9e2e1cff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:47.874147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871183ee-2097-4ec2-b6b5-451c9e2e1cff None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:47.890717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 308/619] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:47.892429 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4bc5c-2770-4408-a7e6-591fa5db3644 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:47.892901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4bc5c-2770-4408-a7e6-591fa5db3644 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:47.893327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4bc5c-2770-4408-a7e6-591fa5db3644 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:47.895154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 309/620] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:47.904360 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4bc5c-2770-4408-a7e6-591fa5db3644 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:47.971143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12069b8-95b6-437c-839a-9dad405bca2b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:48.012739 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24725d89-055e-4421-93e7-c6b7b6478c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:48.013422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24725d89-055e-4421-93e7-c6b7b6478c01 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:48.013731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24725d89-055e-4421-93e7-c6b7b6478c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:48.014048 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24725d89-055e-4421-93e7-c6b7b6478c01 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:48.022808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24725d89-055e-4421-93e7-c6b7b6478c01 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:48.024366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24725d89-055e-4421-93e7-c6b7b6478c01 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:48.035005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4bc5c-2770-4408-a7e6-591fa5db3644 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-24725d89-055e-4421-93e7-c6b7b6478c01 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:48.037262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:48.038231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:48.039236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:48.039236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:48.039236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:48.039236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:48.039236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:48.039236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:48.039236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:48.039236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:48.039236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:00:48.039236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:48.039236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 312/621] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:00:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:48.041229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833a9d0-6b5b-4b4c-a61b-1d860f414563 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:48.041229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833a9d0-6b5b-4b4c-a61b-1d860f414563 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:48.041229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833a9d0-6b5b-4b4c-a61b-1d860f414563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:48.061565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-5833a9d0-6b5b-4b4c-a61b-1d860f414563 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:48.062796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5833a9d0-6b5b-4b4c-a61b-1d860f414563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:48.091037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d72310-e968-4022-a694-6eeb625313a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:48.091749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d72310-e968-4022-a694-6eeb625313a0 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:48.092040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d72310-e968-4022-a694-6eeb625313a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:48.092316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d72310-e968-4022-a694-6eeb625313a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:48.102007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d72310-e968-4022-a694-6eeb625313a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:48.104105 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d72310-e968-4022-a694-6eeb625313a0 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-65d72310-e968-4022-a694-6eeb625313a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:48.117855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:48.124709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:48.125711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:48.125711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:48.125711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:48.125711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:48.125711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:48.125711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:48.125711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:48.125711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:48.125711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:00:48.125711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:48.125711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 310/622] 10.222.0.61 () {56 vars in 1093 bytes} [Wed Sep 7 13:00:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:48.125711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034748b-af3a-4661-88e1-9910b0c85d1b None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:48.125711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034748b-af3a-4661-88e1-9910b0c85d1b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:48.126690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034748b-af3a-4661-88e1-9910b0c85d1b None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:48.128731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 311/623] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:48.129692 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034748b-af3a-4661-88e1-9910b0c85d1b None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:48.136029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:48.167139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 313/624] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:48.167139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833a9d0-6b5b-4b4c-a61b-1d860f414563 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:48.217479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:48.287156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833a9d0-6b5b-4b4c-a61b-1d860f414563 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:48.291233 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034748b-af3a-4661-88e1-9910b0c85d1b None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:48.328904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48f36c-355a-4bc5-8bba-6fb1cd5ab4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:48.329706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48f36c-355a-4bc5-8bba-6fb1cd5ab4cb None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:48.330049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48f36c-355a-4bc5-8bba-6fb1cd5ab4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:48.330388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48f36c-355a-4bc5-8bba-6fb1cd5ab4cb None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:48.331257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48f36c-355a-4bc5-8bba-6fb1cd5ab4cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:48.331630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48f36c-355a-4bc5-8bba-6fb1cd5ab4cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:48.333351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48f36c-355a-4bc5-8bba-6fb1cd5ab4cb None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:48.341015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b188ad6-7567-4503-a43e-404706d74df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:48.342070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b188ad6-7567-4503-a43e-404706d74df7 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:48.342424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b188ad6-7567-4503-a43e-404706d74df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:48.342774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b188ad6-7567-4503-a43e-404706d74df7 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:48.344004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b188ad6-7567-4503-a43e-404706d74df7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:48.345421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b188ad6-7567-4503-a43e-404706d74df7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:48.347041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b188ad6-7567-4503-a43e-404706d74df7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:48.359637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 314/625] 10.222.0.61 () {56 vars in 1133 bytes} [Wed Sep 7 13:00:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:48.361200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79097fd8-4ea5-49e0-9d49-3edc51d6e169 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:48.361552 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79097fd8-4ea5-49e0-9d49-3edc51d6e169 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:48.361887 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79097fd8-4ea5-49e0-9d49-3edc51d6e169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:48.372717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 312/626] 10.222.0.61 () {56 vars in 1135 bytes} [Wed Sep 7 13:00:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:48.373795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b188ad6-7567-4503-a43e-404706d74df7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:48.386745 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-79097fd8-4ea5-49e0-9d49-3edc51d6e169 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:48.389036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-79097fd8-4ea5-49e0-9d49-3edc51d6e169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:48.470891 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:48.516977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 315/627] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:48.517882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79097fd8-4ea5-49e0-9d49-3edc51d6e169 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:48.528854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b188ad6-7567-4503-a43e-404706d74df7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:48.545056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:48.576723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799aaaf-6343-40b2-8985-b16a4d33a03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:48.578044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799aaaf-6343-40b2-8985-b16a4d33a03f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:48.578618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799aaaf-6343-40b2-8985-b16a4d33a03f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:48.579135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799aaaf-6343-40b2-8985-b16a4d33a03f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:48.660111 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79097fd8-4ea5-49e0-9d49-3edc51d6e169 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:48.672954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799aaaf-6343-40b2-8985-b16a4d33a03f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:48.675016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799aaaf-6343-40b2-8985-b16a4d33a03f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:48.703306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e76fe1-b1aa-45ef-9d4f-c419dd95669a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:48.704006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e76fe1-b1aa-45ef-9d4f-c419dd95669a None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:48.704301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e76fe1-b1aa-45ef-9d4f-c419dd95669a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:48.704582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e76fe1-b1aa-45ef-9d4f-c419dd95669a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:48.717162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e76fe1-b1aa-45ef-9d4f-c419dd95669a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:48.718734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e76fe1-b1aa-45ef-9d4f-c419dd95669a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-59e76fe1-b1aa-45ef-9d4f-c419dd95669a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:48.736886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:48.738121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:48.739286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:48.739286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:48.739286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:48.745751 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 316/628] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:48.747005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e76fe1-b1aa-45ef-9d4f-c419dd95669a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:48.781022 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:48.870210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e76fe1-b1aa-45ef-9d4f-c419dd95669a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:48.888608 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 313/629] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:48.889837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799aaaf-6343-40b2-8985-b16a4d33a03f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:48.905126 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:48.928275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9dd92f-07d1-47e7-b55c-a3c203357694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:48.928920 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dd92f-07d1-47e7-b55c-a3c203357694 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:48.929207 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dd92f-07d1-47e7-b55c-a3c203357694 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:48.929489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dd92f-07d1-47e7-b55c-a3c203357694 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:48.943430 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9dd92f-07d1-47e7-b55c-a3c203357694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-ee9dd92f-07d1-47e7-b55c-a3c203357694 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:48.949544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:48.951546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:48.953229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:48.953229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:48.953229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:48.953229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:48.953229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:48.953229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:48.953229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:48.953229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:48.953229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:48.953229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:48.953229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:48.953229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:48.955449 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 317/630] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:48.956894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9dd92f-07d1-47e7-b55c-a3c203357694 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:49.054446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799aaaf-6343-40b2-8985-b16a4d33a03f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:49.091300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:49.108698 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9dd92f-07d1-47e7-b55c-a3c203357694 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:49.110318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcb686-381f-4135-a0c4-1a79da4cccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:49.111407 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcb686-381f-4135-a0c4-1a79da4cccf7 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:49.111888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcb686-381f-4135-a0c4-1a79da4cccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:49.112341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcb686-381f-4135-a0c4-1a79da4cccf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:49.113483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcb686-381f-4135-a0c4-1a79da4cccf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:49.113990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcb686-381f-4135-a0c4-1a79da4cccf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:49.116552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcb686-381f-4135-a0c4-1a79da4cccf7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:49.138435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 314/631] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:49.139614 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcb686-381f-4135-a0c4-1a79da4cccf7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:49.149455 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679c0b2-ddb8-4d1d-9583-bf50665d947c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:49.150107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679c0b2-ddb8-4d1d-9583-bf50665d947c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:49.150410 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679c0b2-ddb8-4d1d-9583-bf50665d947c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:49.150668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679c0b2-ddb8-4d1d-9583-bf50665d947c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:49.163421 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679c0b2-ddb8-4d1d-9583-bf50665d947c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-a679c0b2-ddb8-4d1d-9583-bf50665d947c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:49.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:49.170475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 318/632] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:49.172129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679c0b2-ddb8-4d1d-9583-bf50665d947c None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:49.265101 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:49.303444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679c0b2-ddb8-4d1d-9583-bf50665d947c None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:49.311659 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcb686-381f-4135-a0c4-1a79da4cccf7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:49.348824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a0aee-d9ba-40c6-92e6-68f1526e895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:49.349798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a0aee-d9ba-40c6-92e6-68f1526e895e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:49.350273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a0aee-d9ba-40c6-92e6-68f1526e895e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:49.350641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a0aee-d9ba-40c6-92e6-68f1526e895e None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:49.351845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a0aee-d9ba-40c6-92e6-68f1526e895e None admin] RBAC: Adding query filter params (name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:49.352869 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a0aee-d9ba-40c6-92e6-68f1526e895e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:49.355378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a0aee-d9ba-40c6-92e6-68f1526e895e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:49.358425 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65660479-ced2-4eff-a8fe-f60603f25bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:49.359428 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65660479-ced2-4eff-a8fe-f60603f25bef None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:49.359757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65660479-ced2-4eff-a8fe-f60603f25bef None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:49.360192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65660479-ced2-4eff-a8fe-f60603f25bef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:49.376681 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 319/633] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:49.376681 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915306d9-38e9-4f99-ad7a-94610e374c1c None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:49.376681 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915306d9-38e9-4f99-ad7a-94610e374c1c None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:49.376681 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915306d9-38e9-4f99-ad7a-94610e374c1c None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:49.376681 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 320/634] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:49.377662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915306d9-38e9-4f99-ad7a-94610e374c1c None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:49.382292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65660479-ced2-4eff-a8fe-f60603f25bef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:49.384568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65660479-ced2-4eff-a8fe-f60603f25bef None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:49.398056 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-65660479-ced2-4eff-a8fe-f60603f25bef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:49.405378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:49.407439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:49.409301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:49.409301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:49.409301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:49.409301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 315/635] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:49.409301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65660479-ced2-4eff-a8fe-f60603f25bef None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:49.521146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915306d9-38e9-4f99-ad7a-94610e374c1c None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:49.546441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65660479-ced2-4eff-a8fe-f60603f25bef None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:49.569489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd608c-b2bb-45cc-b2b6-7120e1d7fbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:49.570247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd608c-b2bb-45cc-b2b6-7120e1d7fbbd None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:49.570588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd608c-b2bb-45cc-b2b6-7120e1d7fbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:49.570974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd608c-b2bb-45cc-b2b6-7120e1d7fbbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:49.577061 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:49.580251 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd608c-b2bb-45cc-b2b6-7120e1d7fbbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:49.582280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd608c-b2bb-45cc-b2b6-7120e1d7fbbd None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:49.591614 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9588ba-36dd-4e18-8f85-def5ea96eb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:49.592315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9588ba-36dd-4e18-8f85-def5ea96eb08 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:49.592657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9588ba-36dd-4e18-8f85-def5ea96eb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:49.592988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9588ba-36dd-4e18-8f85-def5ea96eb08 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:49.593756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9588ba-36dd-4e18-8f85-def5ea96eb08 None admin] RBAC: Adding query filter params (name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:49.594120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9588ba-36dd-4e18-8f85-def5ea96eb08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:49.595735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9588ba-36dd-4e18-8f85-def5ea96eb08 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-46cd608c-b2bb-45cc-b2b6-7120e1d7fbbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:49.596514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:49.598300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:49.600620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:49.600620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:49.600620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:49.600620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:49.600620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:49.600620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:49.600620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:49.600620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:49.600620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 321/636] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:49.600620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd608c-b2bb-45cc-b2b6-7120e1d7fbbd None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:49.612089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 316/637] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:49.614310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d50849-b1f6-46ed-939e-5439c502c15a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:49.614791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d50849-b1f6-46ed-939e-5439c502c15a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:49.615365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d50849-b1f6-46ed-939e-5439c502c15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:49.639409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f1d50849-b1f6-46ed-939e-5439c502c15a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:49.640865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f1d50849-b1f6-46ed-939e-5439c502c15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:49.712955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:49.763244 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 317/638] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:49.763504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd608c-b2bb-45cc-b2b6-7120e1d7fbbd None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:49.763640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d50849-b1f6-46ed-939e-5439c502c15a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:49.812708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30b8a8-3723-4bb2-8444-d072366aa3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:49.813759 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30b8a8-3723-4bb2-8444-d072366aa3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:49.814253 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30b8a8-3723-4bb2-8444-d072366aa3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:49.814744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30b8a8-3723-4bb2-8444-d072366aa3b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:49.837226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30b8a8-3723-4bb2-8444-d072366aa3b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:49.840075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30b8a8-3723-4bb2-8444-d072366aa3b3 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-9c30b8a8-3723-4bb2-8444-d072366aa3b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:49.864041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:49.865952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:49.867836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:49.867836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:49.867836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:49.867836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 322/639] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:49.867836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30b8a8-3723-4bb2-8444-d072366aa3b3 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:49.917441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d50849-b1f6-46ed-939e-5439c502c15a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:49.937917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:49.966930 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47841308-7398-4040-a27f-9276b803f070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:49.967549 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841308-7398-4040-a27f-9276b803f070 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:49.967823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841308-7398-4040-a27f-9276b803f070 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:49.968105 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841308-7398-4040-a27f-9276b803f070 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:49.968728 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47841308-7398-4040-a27f-9276b803f070 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:49.969204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47841308-7398-4040-a27f-9276b803f070 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:49.970793 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47841308-7398-4040-a27f-9276b803f070 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:49.994565 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 318/640] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:49.994565 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47841308-7398-4040-a27f-9276b803f070 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:50.072109 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30b8a8-3723-4bb2-8444-d072366aa3b3 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:50.106341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:50.123662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa5634-4e22-4e2c-a982-5078e2066f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:50.124428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa5634-4e22-4e2c-a982-5078e2066f22 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:50.124871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa5634-4e22-4e2c-a982-5078e2066f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:50.125312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa5634-4e22-4e2c-a982-5078e2066f22 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:50.126768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa5634-4e22-4e2c-a982-5078e2066f22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:50.127325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa5634-4e22-4e2c-a982-5078e2066f22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:50.129753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa5634-4e22-4e2c-a982-5078e2066f22 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:50.151438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 323/641] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:50.152616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa5634-4e22-4e2c-a982-5078e2066f22 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:50.168350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47841308-7398-4040-a27f-9276b803f070 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:50.206506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5edda-bec0-407f-bbea-3be66c5f8923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:50.207187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5edda-bec0-407f-bbea-3be66c5f8923 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:50.207476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5edda-bec0-407f-bbea-3be66c5f8923 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:50.207748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5edda-bec0-407f-bbea-3be66c5f8923 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:50.221126 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5edda-bec0-407f-bbea-3be66c5f8923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:50.223733 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5edda-bec0-407f-bbea-3be66c5f8923 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-41a5edda-bec0-407f-bbea-3be66c5f8923 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:50.242413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:50.244557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:50.246416 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:50.246416 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:50.246416 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:50.246416 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 319/642] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:50.246416 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5edda-bec0-407f-bbea-3be66c5f8923 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:50.283730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:50.292998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa5634-4e22-4e2c-a982-5078e2066f22 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:50.338331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d98fd-ea6b-4644-b928-f2fee46be7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:50.338945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d98fd-ea6b-4644-b928-f2fee46be7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:50.339255 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d98fd-ea6b-4644-b928-f2fee46be7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:50.339537 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d98fd-ea6b-4644-b928-f2fee46be7d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:50.340198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d98fd-ea6b-4644-b928-f2fee46be7d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:50.340521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d98fd-ea6b-4644-b928-f2fee46be7d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:50.341897 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d98fd-ea6b-4644-b928-f2fee46be7d7 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:50.359413 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 324/643] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:50.360711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d98fd-ea6b-4644-b928-f2fee46be7d7 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:50.383652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5edda-bec0-407f-bbea-3be66c5f8923 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:50.421104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02142364-0574-45f9-858f-3b78ab832efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:50.421757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02142364-0574-45f9-858f-3b78ab832efe None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:50.422170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02142364-0574-45f9-858f-3b78ab832efe None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:50.422383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02142364-0574-45f9-858f-3b78ab832efe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:50.430792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02142364-0574-45f9-858f-3b78ab832efe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:50.432348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02142364-0574-45f9-858f-3b78ab832efe None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:50.441361 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-02142364-0574-45f9-858f-3b78ab832efe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:50.445280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:50.446834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:50.448789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:50.448789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:50.448789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:50.448789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:50.448789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:50.448789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:50.448789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:50.448789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:50.448789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 320/644] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:50.448789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02142364-0574-45f9-858f-3b78ab832efe None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:50.532480 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d98fd-ea6b-4644-b928-f2fee46be7d7 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:50.557252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:50.575216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02142364-0574-45f9-858f-3b78ab832efe None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:50.589067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e13f9-236f-4fd4-b4d6-8b943316e71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:50.590167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e13f9-236f-4fd4-b4d6-8b943316e71f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:50.590721 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e13f9-236f-4fd4-b4d6-8b943316e71f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:50.591244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e13f9-236f-4fd4-b4d6-8b943316e71f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:50.616797 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56983eb0-b705-43c8-8466-c8662d2e607f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:50.617473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56983eb0-b705-43c8-8466-c8662d2e607f None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:50.618331 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56983eb0-b705-43c8-8466-c8662d2e607f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:50.618606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56983eb0-b705-43c8-8466-c8662d2e607f None admin] PATH_INFO: `/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/0f886e0999d34bf9870b448772cc38a3/roles/5f8e244e8257481e85e65a504aa4bb74` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:50.619843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e13f9-236f-4fd4-b4d6-8b943316e71f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-333e13f9-236f-4fd4-b4d6-8b943316e71f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:50.630935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:50.632200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:50.633880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:50.633880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:50.633880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:50.633880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:50.633880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:50.633880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:50.633880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:50.633880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:50.633880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:50.633880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:50.633880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:50.633880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:50.633880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 325/645] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:50.635248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e13f9-236f-4fd4-b4d6-8b943316e71f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:50.640946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56983eb0-b705-43c8-8466-c8662d2e607f None admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user_id=0f886e0999d34bf9870b448772cc38a3, role_id=5f8e244e8257481e85e65a504aa4bb74)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:50.642570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56983eb0-b705-43c8-8466-c8662d2e607f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:50.658438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 321/646] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:00:50 2022] PUT /identity/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/0f886e0999d34bf9870b448772cc38a3/roles/5f8e244e8257481e85e65a504aa4bb74 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:00:50.659650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56983eb0-b705-43c8-8466-c8662d2e607f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:50.825192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56983eb0-b705-43c8-8466-c8662d2e607f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:50.836816 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e13f9-236f-4fd4-b4d6-8b943316e71f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:50.840073 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:50.880761 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a10ad-42ac-42a9-bb92-edb8e3edec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:50.881437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a10ad-42ac-42a9-bb92-edb8e3edec28 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:50.881722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a10ad-42ac-42a9-bb92-edb8e3edec28 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:50.881995 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a10ad-42ac-42a9-bb92-edb8e3edec28 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:50.885655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:50.892473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a10ad-42ac-42a9-bb92-edb8e3edec28 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:50.893949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a10ad-42ac-42a9-bb92-edb8e3edec28 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-da8a10ad-42ac-42a9-bb92-edb8e3edec28 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:50.909218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:50.911312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:50.920188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:50.920188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:50.920188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:50.920188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:50.920188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:50.920188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:50.920188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:50.920188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:50.920188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:00:50.920188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:50.920188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 322/647] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:50.920188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a10ad-42ac-42a9-bb92-edb8e3edec28 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:50.926367 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecada7-b919-492d-a037-145cbd916b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:50.928410 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecada7-b919-492d-a037-145cbd916b78 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:50.928984 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecada7-b919-492d-a037-145cbd916b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:50.929464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecada7-b919-492d-a037-145cbd916b78 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:50.932647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecada7-b919-492d-a037-145cbd916b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:50.934682 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecada7-b919-492d-a037-145cbd916b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:50.937686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecada7-b919-492d-a037-145cbd916b78 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:50.959871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 326/648] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:50.962201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecada7-b919-492d-a037-145cbd916b78 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:51.097937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a10ad-42ac-42a9-bb92-edb8e3edec28 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:51.133980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecada7-b919-492d-a037-145cbd916b78 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:51.171851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd2f0c-a74b-46f4-9889-72ab40a81020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:51.172809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd2f0c-a74b-46f4-9889-72ab40a81020 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:51.173282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd2f0c-a74b-46f4-9889-72ab40a81020 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:51.173745 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd2f0c-a74b-46f4-9889-72ab40a81020 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:51.174986 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd2f0c-a74b-46f4-9889-72ab40a81020 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:51.175561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd2f0c-a74b-46f4-9889-72ab40a81020 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:51.177799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd2f0c-a74b-46f4-9889-72ab40a81020 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:51.191639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918da933-aad9-430e-bf6c-0a943c3a2453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:51.192463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918da933-aad9-430e-bf6c-0a943c3a2453 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:51.192877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918da933-aad9-430e-bf6c-0a943c3a2453 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:51.193284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918da933-aad9-430e-bf6c-0a943c3a2453 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:51.194118 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918da933-aad9-430e-bf6c-0a943c3a2453 None admin] RBAC: Adding query filter params (name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:51.194573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918da933-aad9-430e-bf6c-0a943c3a2453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:51.196567 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918da933-aad9-430e-bf6c-0a943c3a2453 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:51.210027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 323/649] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:51.210027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd2f0c-a74b-46f4-9889-72ab40a81020 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:51.210457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 327/650] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:51.212545 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918da933-aad9-430e-bf6c-0a943c3a2453 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:51.265513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:51.318008 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:51.397220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd2f0c-a74b-46f4-9889-72ab40a81020 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:51.400907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918da933-aad9-430e-bf6c-0a943c3a2453 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:51.448013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0964603-66b8-4873-9e24-5500fcd7e49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:51.448769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0964603-66b8-4873-9e24-5500fcd7e49f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:51.449120 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0964603-66b8-4873-9e24-5500fcd7e49f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:51.449470 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0964603-66b8-4873-9e24-5500fcd7e49f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:51.462281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0964603-66b8-4873-9e24-5500fcd7e49f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:51.464863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0964603-66b8-4873-9e24-5500fcd7e49f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:51.473416 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55be078-5918-4677-95ea-c0b91d40aec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:51.474789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55be078-5918-4677-95ea-c0b91d40aec7 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:51.475547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55be078-5918-4677-95ea-c0b91d40aec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:51.476267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55be078-5918-4677-95ea-c0b91d40aec7 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:51.478389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55be078-5918-4677-95ea-c0b91d40aec7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:51.479235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55be078-5918-4677-95ea-c0b91d40aec7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:51.482146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55be078-5918-4677-95ea-c0b91d40aec7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-e0964603-66b8-4873-9e24-5500fcd7e49f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:51.487516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:51.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:51.491849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:51.491849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:51.491849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:51.491849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:51.491849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:51.491849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:51.491849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:51.491849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:51.491849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 13:00:51.491849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:51.496051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 328/651] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:51.497585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0964603-66b8-4873-9e24-5500fcd7e49f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:51.522372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 324/652] 10.222.0.61 () {56 vars in 1137 bytes} [Wed Sep 7 13:00:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:51.524484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55be078-5918-4677-95ea-c0b91d40aec7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:51.680634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:51.688444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55be078-5918-4677-95ea-c0b91d40aec7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:51.712104 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0964603-66b8-4873-9e24-5500fcd7e49f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:51.716614 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:51.755729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050ff00-875b-475c-becb-6ff88e6f10c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:51.756461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ff00-875b-475c-becb-6ff88e6f10c2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:51.756775 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ff00-875b-475c-becb-6ff88e6f10c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:51.757095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ff00-875b-475c-becb-6ff88e6f10c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:51.772071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050ff00-875b-475c-becb-6ff88e6f10c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:51.773879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050ff00-875b-475c-becb-6ff88e6f10c2 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:51.786133 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f950c-3d43-404a-a554-6e9a8b2396d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:51.787164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f950c-3d43-404a-a554-6e9a8b2396d4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:51.787650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f950c-3d43-404a-a554-6e9a8b2396d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:51.788104 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f950c-3d43-404a-a554-6e9a8b2396d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:51.789046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f950c-3d43-404a-a554-6e9a8b2396d4 None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=ba93e2d4aaed4c5595d5fd7363626bcb) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:51.790073 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f950c-3d43-404a-a554-6e9a8b2396d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-d050ff00-875b-475c-becb-6ff88e6f10c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:51.793290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:51.794390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:51.795446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:51.795446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:51.795446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:51.796790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f950c-3d43-404a-a554-6e9a8b2396d4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:51.799261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 325/653] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:51.800255 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050ff00-875b-475c-becb-6ff88e6f10c2 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:51.806930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 329/654] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:00:51 2022] GET /identity/v3/role_assignments?user.id=ba93e2d4aaed4c5595d5fd7363626bcb&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:51.808200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f950c-3d43-404a-a554-6e9a8b2396d4 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:51.936184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f950c-3d43-404a-a554-6e9a8b2396d4 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:51.967298 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050ff00-875b-475c-becb-6ff88e6f10c2 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:51.978847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0419b7-817a-4ddd-9fb2-fac6ca826e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:51.979517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0419b7-817a-4ddd-9fb2-fac6ca826e1f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:51.979801 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0419b7-817a-4ddd-9fb2-fac6ca826e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:51.980080 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0419b7-817a-4ddd-9fb2-fac6ca826e1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:52.003085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:52.048091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1cabf-bc23-43cf-b26a-76c2e4b0aaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:52.049702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1cabf-bc23-43cf-b26a-76c2e4b0aaf9 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:52.050460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1cabf-bc23-43cf-b26a-76c2e4b0aaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:52.051196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1cabf-bc23-43cf-b26a-76c2e4b0aaf9 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:52.053240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1cabf-bc23-43cf-b26a-76c2e4b0aaf9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:52.054076 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1cabf-bc23-43cf-b26a-76c2e4b0aaf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:52.057473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1cabf-bc23-43cf-b26a-76c2e4b0aaf9 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:52.072487 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0419b7-817a-4ddd-9fb2-fac6ca826e1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:52.075369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0419b7-817a-4ddd-9fb2-fac6ca826e1f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:52.089963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 326/655] 10.222.0.61 () {56 vars in 1137 bytes} [Wed Sep 7 13:00:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:52.090891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1cabf-bc23-43cf-b26a-76c2e4b0aaf9 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:52.138464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:52.239565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 330/656] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:52.241321 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0419b7-817a-4ddd-9fb2-fac6ca826e1f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:52.287947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1cabf-bc23-43cf-b26a-76c2e4b0aaf9 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:52.325056 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:52.366315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975fd35-d94f-440a-b822-3877f248422f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:52.367426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975fd35-d94f-440a-b822-3877f248422f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:52.367912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975fd35-d94f-440a-b822-3877f248422f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:52.368391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975fd35-d94f-440a-b822-3877f248422f None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:52.370641 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975fd35-d94f-440a-b822-3877f248422f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:52.371213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975fd35-d94f-440a-b822-3877f248422f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:52.374847 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975fd35-d94f-440a-b822-3877f248422f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:52.392815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 327/657] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:52.393945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975fd35-d94f-440a-b822-3877f248422f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:52.414245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0419b7-817a-4ddd-9fb2-fac6ca826e1f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:52.459375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0d916-628b-40f2-b96d-be95539984b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:52.460123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0d916-628b-40f2-b96d-be95539984b2 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:52.460291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0d916-628b-40f2-b96d-be95539984b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:52.460575 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0d916-628b-40f2-b96d-be95539984b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:52.515844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:52.540218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975fd35-d94f-440a-b822-3877f248422f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:52.540473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0d916-628b-40f2-b96d-be95539984b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:52.541943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0d916-628b-40f2-b96d-be95539984b2 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:52.587616 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5aab00-df94-4ba0-8a6c-8a5348b4b3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:52.588609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5aab00-df94-4ba0-8a6c-8a5348b4b3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:52.589053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5aab00-df94-4ba0-8a6c-8a5348b4b3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:52.589718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5aab00-df94-4ba0-8a6c-8a5348b4b3a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:52.599157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:52.609990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5aab00-df94-4ba0-8a6c-8a5348b4b3a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:52.612026 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5aab00-df94-4ba0-8a6c-8a5348b4b3a9 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-ce5aab00-df94-4ba0-8a6c-8a5348b4b3a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:52.634703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:52.636179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:52.637236 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:52.637236 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:52.637236 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:52.639265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 328/658] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:52.640345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5aab00-df94-4ba0-8a6c-8a5348b4b3a9 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:52.662808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 331/659] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:52.663828 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0d916-628b-40f2-b96d-be95539984b2 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:52.784369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0d916-628b-40f2-b96d-be95539984b2 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:52.786892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5aab00-df94-4ba0-8a6c-8a5348b4b3a9 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:52.825624 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1db5fa-e808-4e0b-9d27-1ab715af04ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:52.826214 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1db5fa-e808-4e0b-9d27-1ab715af04ca None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:52.826498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1db5fa-e808-4e0b-9d27-1ab715af04ca None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:52.826774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1db5fa-e808-4e0b-9d27-1ab715af04ca None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:52.827780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1db5fa-e808-4e0b-9d27-1ab715af04ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:52.828000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1db5fa-e808-4e0b-9d27-1ab715af04ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:52.829442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1db5fa-e808-4e0b-9d27-1ab715af04ca None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:52.838119 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba4d53-2074-47fe-9992-62398eef02a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:52.839595 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba4d53-2074-47fe-9992-62398eef02a4 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:52.840139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba4d53-2074-47fe-9992-62398eef02a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:52.840647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba4d53-2074-47fe-9992-62398eef02a4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:52.843421 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 332/660] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:52.844188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1db5fa-e808-4e0b-9d27-1ab715af04ca None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:52.845475 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:52.854642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba4d53-2074-47fe-9992-62398eef02a4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:52.856509 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba4d53-2074-47fe-9992-62398eef02a4 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:52.857243 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-3dba4d53-2074-47fe-9992-62398eef02a4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:52.874654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:52.875806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:52.877299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:52.877299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:52.877299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:52.877299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:52.877299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:52.877299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:52.877299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:52.877299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:52.877299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 13:00:52.877299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:52.885554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 329/661] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:52.886991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba4d53-2074-47fe-9992-62398eef02a4 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:52.965767 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1db5fa-e808-4e0b-9d27-1ab715af04ca None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:53.006377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42547e7-b974-46ff-ab92-b38de0d50a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:53.007038 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42547e7-b974-46ff-ab92-b38de0d50a45 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:53.007371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42547e7-b974-46ff-ab92-b38de0d50a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:53.007974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42547e7-b974-46ff-ab92-b38de0d50a45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:53.020860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42547e7-b974-46ff-ab92-b38de0d50a45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:53.022381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42547e7-b974-46ff-ab92-b38de0d50a45 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:53.029306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba4d53-2074-47fe-9992-62398eef02a4 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-e42547e7-b974-46ff-ab92-b38de0d50a45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:53.040497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:53.041566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:53.042536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:53.042536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:53.042536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:53.042536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 333/662] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:53.042794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42547e7-b974-46ff-ab92-b38de0d50a45 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:53.086808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2854a65-7728-498d-b061-c1c3321f9a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:53.087976 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2854a65-7728-498d-b061-c1c3321f9a04 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:53.088465 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2854a65-7728-498d-b061-c1c3321f9a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:53.088937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2854a65-7728-498d-b061-c1c3321f9a04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:53.098286 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2854a65-7728-498d-b061-c1c3321f9a04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:53.100805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2854a65-7728-498d-b061-c1c3321f9a04 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:53.113684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-e2854a65-7728-498d-b061-c1c3321f9a04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:53.120253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:53.121204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:53.122102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:53.122102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:53.122102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:53.122102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:53.122102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:53.122102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:53.122102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:53.122102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:53.124975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 330/663] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:53.126882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2854a65-7728-498d-b061-c1c3321f9a04 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:53.199684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42547e7-b974-46ff-ab92-b38de0d50a45 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:53.218585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:53.281589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866eda33-a2c6-4491-98e3-3afac2e0c58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:53.282899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eda33-a2c6-4491-98e3-3afac2e0c58b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:53.283334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eda33-a2c6-4491-98e3-3afac2e0c58b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:53.283731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eda33-a2c6-4491-98e3-3afac2e0c58b None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:53.284993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866eda33-a2c6-4491-98e3-3afac2e0c58b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:53.285459 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866eda33-a2c6-4491-98e3-3afac2e0c58b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:53.290068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866eda33-a2c6-4491-98e3-3afac2e0c58b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:53.292960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2854a65-7728-498d-b061-c1c3321f9a04 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:53.319798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 334/664] 10.222.0.61 () {56 vars in 1139 bytes} [Wed Sep 7 13:00:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:53.319798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866eda33-a2c6-4491-98e3-3afac2e0c58b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:53.370209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240e9ee-1319-4604-9bed-37aa3fead55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:53.371278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240e9ee-1319-4604-9bed-37aa3fead55d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:53.371771 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240e9ee-1319-4604-9bed-37aa3fead55d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:53.372266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240e9ee-1319-4604-9bed-37aa3fead55d None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:53.373459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240e9ee-1319-4604-9bed-37aa3fead55d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:53.373993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240e9ee-1319-4604-9bed-37aa3fead55d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:53.376975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240e9ee-1319-4604-9bed-37aa3fead55d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:53.399569 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 331/665] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:53.400939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240e9ee-1319-4604-9bed-37aa3fead55d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:53.485627 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866eda33-a2c6-4491-98e3-3afac2e0c58b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:53.519352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:53.539259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f3def-e889-4379-81c8-bc3fa85b0cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:53.539926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f3def-e889-4379-81c8-bc3fa85b0cbb None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:53.540215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f3def-e889-4379-81c8-bc3fa85b0cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:53.540491 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f3def-e889-4379-81c8-bc3fa85b0cbb None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:53.541207 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f3def-e889-4379-81c8-bc3fa85b0cbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:53.541518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f3def-e889-4379-81c8-bc3fa85b0cbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:53.543039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f3def-e889-4379-81c8-bc3fa85b0cbb None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:53.558757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240e9ee-1319-4604-9bed-37aa3fead55d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:53.565164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 335/666] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:53.565164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986c5cf-5557-400e-bed3-a458872a1838 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:53.565164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986c5cf-5557-400e-bed3-a458872a1838 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:53.565164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986c5cf-5557-400e-bed3-a458872a1838 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:53.565164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 336/667] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:53.565626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986c5cf-5557-400e-bed3-a458872a1838 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:53.585970 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:53.610176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bcc9e-6deb-49df-83c9-f3e549288739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:53.610803 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bcc9e-6deb-49df-83c9-f3e549288739 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:53.611080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bcc9e-6deb-49df-83c9-f3e549288739 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:53.611383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bcc9e-6deb-49df-83c9-f3e549288739 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:53.707688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bcc9e-6deb-49df-83c9-f3e549288739 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:53.709203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bcc9e-6deb-49df-83c9-f3e549288739 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:53.743818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986c5cf-5557-400e-bed3-a458872a1838 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:53.808322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62026fd7-8968-4e9a-a38d-6d3df8521c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:53.809407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62026fd7-8968-4e9a-a38d-6d3df8521c6e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:53.809873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62026fd7-8968-4e9a-a38d-6d3df8521c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:53.810353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62026fd7-8968-4e9a-a38d-6d3df8521c6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:53.823372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62026fd7-8968-4e9a-a38d-6d3df8521c6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:53.826155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62026fd7-8968-4e9a-a38d-6d3df8521c6e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:53.839831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 332/668] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:53.839831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bcc9e-6deb-49df-83c9-f3e549288739 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-62026fd7-8968-4e9a-a38d-6d3df8521c6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:53.847926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:53.849611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:53.851289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:53.851289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:53.851289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:53.851289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:53.851289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:53.851289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:53.851289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:53.851289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:53.855603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 337/669] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:53.859025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2c532-0e05-43b8-85b4-7de51849a7e3 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:53.859025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2c532-0e05-43b8-85b4-7de51849a7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:53.859025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2c532-0e05-43b8-85b4-7de51849a7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:53.873168 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:53.887145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-40c2c532-0e05-43b8-85b4-7de51849a7e3 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:53.888968 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-40c2c532-0e05-43b8-85b4-7de51849a7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:53.945469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:54.016682 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bcc9e-6deb-49df-83c9-f3e549288739 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:54.017101 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 338/670] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:54.018275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2c532-0e05-43b8-85b4-7de51849a7e3 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:54.098612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e1dd3-b07f-40b9-af32-b9a31dd731ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:54.100169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e1dd3-b07f-40b9-af32-b9a31dd731ef None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:54.100956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e1dd3-b07f-40b9-af32-b9a31dd731ef None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:54.101670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e1dd3-b07f-40b9-af32-b9a31dd731ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:54.103937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e1dd3-b07f-40b9-af32-b9a31dd731ef None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=8356f60a0f684484abc8f8d6fb6a7be6) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:54.104764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e1dd3-b07f-40b9-af32-b9a31dd731ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:54.107850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e1dd3-b07f-40b9-af32-b9a31dd731ef None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:54.125037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 333/671] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:00:53 2022] GET /identity/v3/role_assignments?user.id=8356f60a0f684484abc8f8d6fb6a7be6&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:54.127241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e1dd3-b07f-40b9-af32-b9a31dd731ef None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:54.218176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2c532-0e05-43b8-85b4-7de51849a7e3 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:54.285143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56b273-c673-4252-8e78-727d32b41ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:54.286301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56b273-c673-4252-8e78-727d32b41ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:54.286698 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56b273-c673-4252-8e78-727d32b41ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:54.287192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56b273-c673-4252-8e78-727d32b41ec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:54.325059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56b273-c673-4252-8e78-727d32b41ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:54.327033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56b273-c673-4252-8e78-727d32b41ec0 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-1e56b273-c673-4252-8e78-727d32b41ec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:54.383692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:54.385581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:54.387442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:54.387442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:54.387442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:54.399858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 339/672] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:54.401431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56b273-c673-4252-8e78-727d32b41ec0 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:54.422797 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:54.460811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:54.467554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e1dd3-b07f-40b9-af32-b9a31dd731ef None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:54.543916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b5fff-055a-40c3-8b09-003caccac3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:54.544557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b5fff-055a-40c3-8b09-003caccac3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:54.544919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b5fff-055a-40c3-8b09-003caccac3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:54.545143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b5fff-055a-40c3-8b09-003caccac3e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:54.545830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b5fff-055a-40c3-8b09-003caccac3e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:54.546173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b5fff-055a-40c3-8b09-003caccac3e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:54.548983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b5fff-055a-40c3-8b09-003caccac3e6 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:54.566600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 334/673] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:54.567640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b5fff-055a-40c3-8b09-003caccac3e6 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:54.616350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56b273-c673-4252-8e78-727d32b41ec0 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:54.695135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e1e2e-71bb-449d-b8ba-722e5bd29323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:54.696815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e1e2e-71bb-449d-b8ba-722e5bd29323 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:54.697330 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e1e2e-71bb-449d-b8ba-722e5bd29323 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:54.698085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e1e2e-71bb-449d-b8ba-722e5bd29323 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:54.698988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e1e2e-71bb-449d-b8ba-722e5bd29323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:54.699558 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e1e2e-71bb-449d-b8ba-722e5bd29323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:54.703483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e1e2e-71bb-449d-b8ba-722e5bd29323 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:54.732295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 340/674] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:54.733735 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e1e2e-71bb-449d-b8ba-722e5bd29323 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:54.792579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b5fff-055a-40c3-8b09-003caccac3e6 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:54.866697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0d4b0-c78c-4a0e-b1eb-93f40268dfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:54.868415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0d4b0-c78c-4a0e-b1eb-93f40268dfe8 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:54.868884 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0d4b0-c78c-4a0e-b1eb-93f40268dfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:54.869361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0d4b0-c78c-4a0e-b1eb-93f40268dfe8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:54.886363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:54.892856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0d4b0-c78c-4a0e-b1eb-93f40268dfe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:54.893971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-c9e0d4b0-c78c-4a0e-b1eb-93f40268dfe8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:54.906266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:54.908436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 335/675] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:00:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:54.910089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0d4b0-c78c-4a0e-b1eb-93f40268dfe8 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:54.928998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e1e2e-71bb-449d-b8ba-722e5bd29323 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:54.984636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2dbf8-76aa-482d-a849-68992b511c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:54.985528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2dbf8-76aa-482d-a849-68992b511c94 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:54.985983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2dbf8-76aa-482d-a849-68992b511c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:54.986443 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2dbf8-76aa-482d-a849-68992b511c94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:54.987748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2dbf8-76aa-482d-a849-68992b511c94 None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=b4f82adce75b44c6baf85b6b96dd0728) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:54.988258 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2dbf8-76aa-482d-a849-68992b511c94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:54.990847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2dbf8-76aa-482d-a849-68992b511c94 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:55.000829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 341/676] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:00:54 2022] GET /identity/v3/role_assignments?user.id=b4f82adce75b44c6baf85b6b96dd0728&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:55.003532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f80343-595d-42e4-8666-c3cb8a948da4 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:55.004027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f80343-595d-42e4-8666-c3cb8a948da4 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:55.004513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f80343-595d-42e4-8666-c3cb8a948da4 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:55.011892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 342/677] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:55.011892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f80343-595d-42e4-8666-c3cb8a948da4 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:55.129313 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0d4b0-c78c-4a0e-b1eb-93f40268dfe8 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:55.202838 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f80343-595d-42e4-8666-c3cb8a948da4 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:55.207437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d214c-384c-4210-93fb-f0b01edde643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:55.213012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d214c-384c-4210-93fb-f0b01edde643 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:55.214567 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d214c-384c-4210-93fb-f0b01edde643 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:55.215146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d214c-384c-4210-93fb-f0b01edde643 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:55.234406 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d214c-384c-4210-93fb-f0b01edde643 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:55.238601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d214c-384c-4210-93fb-f0b01edde643 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-ec8d214c-384c-4210-93fb-f0b01edde643 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:55.261643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:55.262600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:55.263518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:55.263518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:55.263518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:55.263518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:55.263518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:55.263518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:55.263518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:55.263518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:55.266179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 336/678] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:55.268038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd0ff8-b315-468d-91b9-7c4f3be834c1 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:55.268613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd0ff8-b315-468d-91b9-7c4f3be834c1 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:55.269090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd0ff8-b315-468d-91b9-7c4f3be834c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:55.278178 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4486f-72a4-406d-ae4a-7d2a97268880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:55.279222 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4486f-72a4-406d-ae4a-7d2a97268880 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:55.279552 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4486f-72a4-406d-ae4a-7d2a97268880 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:55.279977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4486f-72a4-406d-ae4a-7d2a97268880 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:55.280941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4486f-72a4-406d-ae4a-7d2a97268880 None admin] RBAC: Adding query filter params (name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:55.281620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4486f-72a4-406d-ae4a-7d2a97268880 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:55.284141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4486f-72a4-406d-ae4a-7d2a97268880 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:55.293005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-a7bd0ff8-b315-468d-91b9-7c4f3be834c1 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:55.295002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a7bd0ff8-b315-468d-91b9-7c4f3be834c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:55.298482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 343/679] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:00:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:55.299980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4486f-72a4-406d-ae4a-7d2a97268880 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:55.320509 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:55.393180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:55.409844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 337/680] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:55.413273 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd0ff8-b315-468d-91b9-7c4f3be834c1 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:55.475965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4486f-72a4-406d-ae4a-7d2a97268880 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:55.547224 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2f4e1-6a32-49c2-a416-2ce7b8a83042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:55.548484 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2f4e1-6a32-49c2-a416-2ce7b8a83042 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:55.549066 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2f4e1-6a32-49c2-a416-2ce7b8a83042 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:55.549540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2f4e1-6a32-49c2-a416-2ce7b8a83042 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:55.550462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2f4e1-6a32-49c2-a416-2ce7b8a83042 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:55.551003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2f4e1-6a32-49c2-a416-2ce7b8a83042 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:55.552716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2f4e1-6a32-49c2-a416-2ce7b8a83042 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:55.569381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 344/681] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:55.580925 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2f4e1-6a32-49c2-a416-2ce7b8a83042 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:55.647851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd0ff8-b315-468d-91b9-7c4f3be834c1 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:55.721256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:55.771763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c0a44-104d-4e0b-9212-37ec37af1db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:55.772039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0a44-104d-4e0b-9212-37ec37af1db6 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:55.772375 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0a44-104d-4e0b-9212-37ec37af1db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:55.772678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0a44-104d-4e0b-9212-37ec37af1db6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:55.793139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c0a44-104d-4e0b-9212-37ec37af1db6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:55.793139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c0a44-104d-4e0b-9212-37ec37af1db6 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:55.796149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2f4e1-6a32-49c2-a416-2ce7b8a83042 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-d55c0a44-104d-4e0b-9212-37ec37af1db6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:55.823412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:55.825230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:55.826963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:55.826963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:55.826963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:55.826963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 338/682] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:55.828530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c0a44-104d-4e0b-9212-37ec37af1db6 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:55.855636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16db266-af48-4207-b66b-06e7e3ea6569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:55.856269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16db266-af48-4207-b66b-06e7e3ea6569 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:55.856521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16db266-af48-4207-b66b-06e7e3ea6569 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:55.856840 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16db266-af48-4207-b66b-06e7e3ea6569 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:55.857789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16db266-af48-4207-b66b-06e7e3ea6569 None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=de05c08e30dd4fff93f1c97aac9553d2) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:55.858131 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16db266-af48-4207-b66b-06e7e3ea6569 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:55.859856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16db266-af48-4207-b66b-06e7e3ea6569 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:55.868930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 345/683] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:00:55 2022] GET /identity/v3/role_assignments?user.id=de05c08e30dd4fff93f1c97aac9553d2&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:55.870297 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16db266-af48-4207-b66b-06e7e3ea6569 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:55.955757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:56.060114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16db266-af48-4207-b66b-06e7e3ea6569 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:56.107064 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c0a44-104d-4e0b-9212-37ec37af1db6 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:56.146492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:56.164915 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5943fa-4369-4052-add0-09ea252b0b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:56.165611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5943fa-4369-4052-add0-09ea252b0b9a None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:56.165889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5943fa-4369-4052-add0-09ea252b0b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:56.166288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5943fa-4369-4052-add0-09ea252b0b9a None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:56.167273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5943fa-4369-4052-add0-09ea252b0b9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:56.170003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5943fa-4369-4052-add0-09ea252b0b9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:56.172071 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5943fa-4369-4052-add0-09ea252b0b9a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:56.208160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 346/684] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:56.209819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1d55a-ec00-4fc6-bd8e-b0dc2df60568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:56.210650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1d55a-ec00-4fc6-bd8e-b0dc2df60568 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:56.211133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1d55a-ec00-4fc6-bd8e-b0dc2df60568 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:56.211969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1d55a-ec00-4fc6-bd8e-b0dc2df60568 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:56.215176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5943fa-4369-4052-add0-09ea252b0b9a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:56.249478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1d55a-ec00-4fc6-bd8e-b0dc2df60568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-1eb1d55a-ec00-4fc6-bd8e-b0dc2df60568 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:00:56.261145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:56.262907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:56.265041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:56.265041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:56.265041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:56.265041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:56.265041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:00:56.265041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:00:56.265041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:00:56.265041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:00:56.265041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:00:56.265041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:00:56.265041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:00:56.265041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:56.266318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 339/685] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:00:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 439 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:56.273620 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1d55a-ec00-4fc6-bd8e-b0dc2df60568 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:56.365280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5943fa-4369-4052-add0-09ea252b0b9a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:56.419973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93679a70-179a-4059-80a1-1dc849f91457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:56.420840 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93679a70-179a-4059-80a1-1dc849f91457 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:56.421160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93679a70-179a-4059-80a1-1dc849f91457 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:56.421505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93679a70-179a-4059-80a1-1dc849f91457 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:56.436419 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93679a70-179a-4059-80a1-1dc849f91457 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:56.438750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93679a70-179a-4059-80a1-1dc849f91457 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-93679a70-179a-4059-80a1-1dc849f91457 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:56.470988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:56.472880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:56.474818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:56.474818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:56.474818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:56.474818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:56.474818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:56.474818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:56.474818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:56.474818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:56.474818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:00:56.474818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:56.480124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 347/686] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:00:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:56.489716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93679a70-179a-4059-80a1-1dc849f91457 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:56.532800 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1d55a-ec00-4fc6-bd8e-b0dc2df60568 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:56.538867 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:56.618836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203df649-5780-45b0-ae76-3355bb6ad3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:56.619989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203df649-5780-45b0-ae76-3355bb6ad3fe None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:56.622542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203df649-5780-45b0-ae76-3355bb6ad3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:56.623468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203df649-5780-45b0-ae76-3355bb6ad3fe None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:56.624579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203df649-5780-45b0-ae76-3355bb6ad3fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:56.625098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203df649-5780-45b0-ae76-3355bb6ad3fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:56.627037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203df649-5780-45b0-ae76-3355bb6ad3fe None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:56.645830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 340/687] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:00:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:56.654396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203df649-5780-45b0-ae76-3355bb6ad3fe None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:56.675741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:56.783576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93679a70-179a-4059-80a1-1dc849f91457 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:56.893264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3694f07-794d-4ecb-aad5-0c970a8a2c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:56.894230 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3694f07-794d-4ecb-aad5-0c970a8a2c23 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:56.894727 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3694f07-794d-4ecb-aad5-0c970a8a2c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:56.895228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3694f07-794d-4ecb-aad5-0c970a8a2c23 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:56.896606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3694f07-794d-4ecb-aad5-0c970a8a2c23 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:56.897156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3694f07-794d-4ecb-aad5-0c970a8a2c23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:56.900554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3694f07-794d-4ecb-aad5-0c970a8a2c23 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:56.942225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 348/688] 10.222.0.61 () {56 vars in 1135 bytes} [Wed Sep 7 13:00:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:00:56.952623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3694f07-794d-4ecb-aad5-0c970a8a2c23 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:56.959993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203df649-5780-45b0-ae76-3355bb6ad3fe None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:57.043011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b9df9-c821-4d04-ad93-0b8a66aaa39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:57.044244 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b9df9-c821-4d04-ad93-0b8a66aaa39a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:57.044744 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b9df9-c821-4d04-ad93-0b8a66aaa39a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:57.045245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b9df9-c821-4d04-ad93-0b8a66aaa39a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:57.079223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b9df9-c821-4d04-ad93-0b8a66aaa39a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:57.082046 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b9df9-c821-4d04-ad93-0b8a66aaa39a None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-e57b9df9-c821-4d04-ad93-0b8a66aaa39a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:57.142447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:57.143749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:57.144989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:57.144989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:57.144989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:57.147302 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 341/689] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 493 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:57.156215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b9df9-c821-4d04-ad93-0b8a66aaa39a None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:57.223650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:57.260270 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3694f07-794d-4ecb-aad5-0c970a8a2c23 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:57.291696 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:57.388726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b32666-ce91-4608-8554-052271c95db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:57.389796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b32666-ce91-4608-8554-052271c95db8 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:57.390266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b32666-ce91-4608-8554-052271c95db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:57.390742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b32666-ce91-4608-8554-052271c95db8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:57.492409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b9df9-c821-4d04-ad93-0b8a66aaa39a None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:57.544280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b32666-ce91-4608-8554-052271c95db8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:57.545767 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b32666-ce91-4608-8554-052271c95db8 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:57.582645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5933e0-cb54-46a8-9038-1a56fd118915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:57.584948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5933e0-cb54-46a8-9038-1a56fd118915 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:57.585765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5933e0-cb54-46a8-9038-1a56fd118915 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:57.587136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5933e0-cb54-46a8-9038-1a56fd118915 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:57.588605 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5933e0-cb54-46a8-9038-1a56fd118915 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:57.590125 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5933e0-cb54-46a8-9038-1a56fd118915 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:57.602172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5933e0-cb54-46a8-9038-1a56fd118915 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:57.657749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 342/690] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:57.666778 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5933e0-cb54-46a8-9038-1a56fd118915 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:57.756369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 349/691] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 805 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:57.759388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21648a7a-5aa9-42a1-9374-1a0498f75b7b None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:57.759993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21648a7a-5aa9-42a1-9374-1a0498f75b7b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:57.760671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21648a7a-5aa9-42a1-9374-1a0498f75b7b None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:57.770082 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 350/692] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:57 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:57.771876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21648a7a-5aa9-42a1-9374-1a0498f75b7b None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:57.789881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:57.835518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:57.886530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5933e0-cb54-46a8-9038-1a56fd118915 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:57.957779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751af15f-916d-42d2-a97e-517e5a0acc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:57.959400 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751af15f-916d-42d2-a97e-517e5a0acc1d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:57.960152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751af15f-916d-42d2-a97e-517e5a0acc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:57.961637 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751af15f-916d-42d2-a97e-517e5a0acc1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:57.974841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751af15f-916d-42d2-a97e-517e5a0acc1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:57.976396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751af15f-916d-42d2-a97e-517e5a0acc1d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:57.978179 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21648a7a-5aa9-42a1-9374-1a0498f75b7b None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-751af15f-916d-42d2-a97e-517e5a0acc1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:00:57.992756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:57.994965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:57.997671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:57.997671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:57.997671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:00:57.997671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:00:57.997671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:00:57.997671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:00:57.997671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:00:57.997671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:00:57.997671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:00:57.997671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:57.997671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 343/693] 10.222.0.61 () {56 vars in 1093 bytes} [Wed Sep 7 13:00:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:57.997671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cdee5-4a24-4b0c-a2e6-14ab48cb5c10 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:57.997671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cdee5-4a24-4b0c-a2e6-14ab48cb5c10 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:58.002153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cdee5-4a24-4b0c-a2e6-14ab48cb5c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:58.021717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-6a4cdee5-4a24-4b0c-a2e6-14ab48cb5c10 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:58.023547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6a4cdee5-4a24-4b0c-a2e6-14ab48cb5c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:58.052785 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91c1b9-a2c4-4be5-9cb6-9c0d17e5349f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:58.053710 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91c1b9-a2c4-4be5-9cb6-9c0d17e5349f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:58.054090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91c1b9-a2c4-4be5-9cb6-9c0d17e5349f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:58.054476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91c1b9-a2c4-4be5-9cb6-9c0d17e5349f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:58.076191 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91c1b9-a2c4-4be5-9cb6-9c0d17e5349f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:58.078183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91c1b9-a2c4-4be5-9cb6-9c0d17e5349f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-0d91c1b9-a2c4-4be5-9cb6-9c0d17e5349f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:58.099933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:58.101708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:58.104117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:58.104117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:58.104117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:00:58.104117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 351/694] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:58.104117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91c1b9-a2c4-4be5-9cb6-9c0d17e5349f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:58.193756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:58.211507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 344/695] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:58.215092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cdee5-4a24-4b0c-a2e6-14ab48cb5c10 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:58.290431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91c1b9-a2c4-4be5-9cb6-9c0d17e5349f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:58.333149 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:58.372429 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6d0f4-544e-491c-813c-287f0432439e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:58.374639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6d0f4-544e-491c-813c-287f0432439e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:58.375487 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6d0f4-544e-491c-813c-287f0432439e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:58.376483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6d0f4-544e-491c-813c-287f0432439e None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:58.379259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6d0f4-544e-491c-813c-287f0432439e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:58.380246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6d0f4-544e-491c-813c-287f0432439e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:58.389732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6d0f4-544e-491c-813c-287f0432439e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:58.404043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cdee5-4a24-4b0c-a2e6-14ab48cb5c10 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:58.436856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 352/696] 10.222.0.61 () {56 vars in 1133 bytes} [Wed Sep 7 13:00:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:58.445489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6d0f4-544e-491c-813c-287f0432439e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:58.471958 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af58707-4dd9-4e66-a7c9-49ec64c7cda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:58.472599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af58707-4dd9-4e66-a7c9-49ec64c7cda7 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:58.472878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af58707-4dd9-4e66-a7c9-49ec64c7cda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:58.473205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af58707-4dd9-4e66-a7c9-49ec64c7cda7 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:58.474107 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af58707-4dd9-4e66-a7c9-49ec64c7cda7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:58.474415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af58707-4dd9-4e66-a7c9-49ec64c7cda7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:58.476015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af58707-4dd9-4e66-a7c9-49ec64c7cda7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:58.506626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 345/697] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:58.515516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af58707-4dd9-4e66-a7c9-49ec64c7cda7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:58.658692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6d0f4-544e-491c-813c-287f0432439e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:58.707741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:58.730340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbfc8b5-c305-435d-beab-7e0a16fc22fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:58.730978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfc8b5-c305-435d-beab-7e0a16fc22fd None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:58.731323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfc8b5-c305-435d-beab-7e0a16fc22fd None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:58.731605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfc8b5-c305-435d-beab-7e0a16fc22fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:58.823958 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbfc8b5-c305-435d-beab-7e0a16fc22fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:58.825609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbfc8b5-c305-435d-beab-7e0a16fc22fd None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:58.908399 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af58707-4dd9-4e66-a7c9-49ec64c7cda7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:58.955589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:58.970181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd1c85-1477-4de4-8d39-03955c5b624f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:58.971931 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd1c85-1477-4de4-8d39-03955c5b624f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:58.972562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd1c85-1477-4de4-8d39-03955c5b624f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:58.973004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd1c85-1477-4de4-8d39-03955c5b624f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:58.982824 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd1c85-1477-4de4-8d39-03955c5b624f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:58.984747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd1c85-1477-4de4-8d39-03955c5b624f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-78dd1c85-1477-4de4-8d39-03955c5b624f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:00:59.002952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:59.004189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:59.005077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:00:59.005077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:00:59.005077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:00:59.005077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:00:59.005077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:00:59.005077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:00:59.005077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:00:59.005077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:59.007544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 346/698] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:00:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 493 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:00:59.009749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd1c85-1477-4de4-8d39-03955c5b624f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:59.038325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 353/699] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:00:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:00:59.041194 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362c5f2-e867-4755-b810-488eac96fa7c None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:59.041676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362c5f2-e867-4755-b810-488eac96fa7c None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:59.042163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362c5f2-e867-4755-b810-488eac96fa7c None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:59.049474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 354/700] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:59.049474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362c5f2-e867-4755-b810-488eac96fa7c None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:59.076417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:59.180511 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd1c85-1477-4de4-8d39-03955c5b624f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:59.219211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362c5f2-e867-4755-b810-488eac96fa7c None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:59.248681 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530536ce-eac4-42db-9e7f-a3a496c71903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:59.250093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530536ce-eac4-42db-9e7f-a3a496c71903 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:59.250625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530536ce-eac4-42db-9e7f-a3a496c71903 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:59.251571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530536ce-eac4-42db-9e7f-a3a496c71903 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:59.253728 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530536ce-eac4-42db-9e7f-a3a496c71903 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:59.258474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530536ce-eac4-42db-9e7f-a3a496c71903 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:59.279735 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef36e89-89b2-459e-bbf4-1e84d4293de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:59.280734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef36e89-89b2-459e-bbf4-1e84d4293de2 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:59.281213 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef36e89-89b2-459e-bbf4-1e84d4293de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:59.281679 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef36e89-89b2-459e-bbf4-1e84d4293de2 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:59.282942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef36e89-89b2-459e-bbf4-1e84d4293de2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:59.283461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef36e89-89b2-459e-bbf4-1e84d4293de2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:59.286076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef36e89-89b2-459e-bbf4-1e84d4293de2 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-530536ce-eac4-42db-9e7f-a3a496c71903 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:59.287928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:59.289480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:00:59.290414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:59.291835 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 347/701] 10.222.0.61 () {56 vars in 1129 bytes} [Wed Sep 7 13:00:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:59.292983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530536ce-eac4-42db-9e7f-a3a496c71903 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:59.317992 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 355/702] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:00:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:59.317992 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3c8f2-bfd3-4636-8616-4f72efba557f None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:59.317992 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3c8f2-bfd3-4636-8616-4f72efba557f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:59.317992 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3c8f2-bfd3-4636-8616-4f72efba557f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:59.341795 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-8fe3c8f2-bfd3-4636-8616-4f72efba557f None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:00:59.343833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-8fe3c8f2-bfd3-4636-8616-4f72efba557f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:00:59.410955 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:59.467184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 356/703] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:00:59.467184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3c8f2-bfd3-4636-8616-4f72efba557f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:59.488422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:59.496757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530536ce-eac4-42db-9e7f-a3a496c71903 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:59.545144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a75436-b718-4b2f-b3e7-f2618dfbb42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:59.546250 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a75436-b718-4b2f-b3e7-f2618dfbb42c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:59.546749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a75436-b718-4b2f-b3e7-f2618dfbb42c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:59.547456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a75436-b718-4b2f-b3e7-f2618dfbb42c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:59.561083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a75436-b718-4b2f-b3e7-f2618dfbb42c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:59.562608 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a75436-b718-4b2f-b3e7-f2618dfbb42c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-53a75436-b718-4b2f-b3e7-f2618dfbb42c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:00:59.579287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:00:59.581238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:00:59.582907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:00:59.582907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:00:59.582907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:00:59.582907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 348/704] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:00:59.582907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a75436-b718-4b2f-b3e7-f2618dfbb42c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:59.629690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3c8f2-bfd3-4636-8616-4f72efba557f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:59.688100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c80dfb-1e4a-4ae2-b98f-940f956e50c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:59.689080 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c80dfb-1e4a-4ae2-b98f-940f956e50c6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:59.689493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c80dfb-1e4a-4ae2-b98f-940f956e50c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:59.689903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c80dfb-1e4a-4ae2-b98f-940f956e50c6 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:59.690887 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c80dfb-1e4a-4ae2-b98f-940f956e50c6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:59.691370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c80dfb-1e4a-4ae2-b98f-940f956e50c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:59.693135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c80dfb-1e4a-4ae2-b98f-940f956e50c6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:59.699827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 357/705] 10.222.0.61 () {56 vars in 1124 bytes} [Wed Sep 7 13:00:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:59.701253 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d84e6-636e-4b13-ab8c-ef85d1d30686 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:59.701666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d84e6-636e-4b13-ab8c-ef85d1d30686 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:59.702066 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d84e6-636e-4b13-ab8c-ef85d1d30686 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:59.704268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 358/706] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:00:59.709717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d84e6-636e-4b13-ab8c-ef85d1d30686 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:59.741622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a75436-b718-4b2f-b3e7-f2618dfbb42c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:59.744586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:59.806380 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64143e0e-5e31-45b8-9020-30e8aeb07d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:59.807351 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64143e0e-5e31-45b8-9020-30e8aeb07d9d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:59.807793 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64143e0e-5e31-45b8-9020-30e8aeb07d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:59.808329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64143e0e-5e31-45b8-9020-30e8aeb07d9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:59.809503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64143e0e-5e31-45b8-9020-30e8aeb07d9d None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=0f886e0999d34bf9870b448772cc38a3) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:59.810010 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64143e0e-5e31-45b8-9020-30e8aeb07d9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:59.812487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64143e0e-5e31-45b8-9020-30e8aeb07d9d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:59.832657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:00:59.833689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 349/707] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:00:59 2022] GET /identity/v3/role_assignments?user.id=0f886e0999d34bf9870b448772cc38a3&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:59.835832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64143e0e-5e31-45b8-9020-30e8aeb07d9d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:00:59.897352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d84e6-636e-4b13-ab8c-ef85d1d30686 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:00:59.952034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7183241-7f21-468b-8815-947a995aa6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:00:59.952957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7183241-7f21-468b-8815-947a995aa6eb None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:59.953382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7183241-7f21-468b-8815-947a995aa6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:59.953800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7183241-7f21-468b-8815-947a995aa6eb None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:59.954851 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7183241-7f21-468b-8815-947a995aa6eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:00:59.955349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7183241-7f21-468b-8815-947a995aa6eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:00:59.957686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7183241-7f21-468b-8815-947a995aa6eb None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:00:59.983347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 359/708] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:00:59.983347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f02a8a-cd1e-407b-b306-f9eea188cdfb None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:00:59.983347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f02a8a-cd1e-407b-b306-f9eea188cdfb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:00:59.983347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f02a8a-cd1e-407b-b306-f9eea188cdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:00:59.995080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64143e0e-5e31-45b8-9020-30e8aeb07d9d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:00.013095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-89f02a8a-cd1e-407b-b306-f9eea188cdfb None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:00.015065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-89f02a8a-cd1e-407b-b306-f9eea188cdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:00.049337 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6a9aa-2faa-4d7d-a964-49224c9bc8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:00.050066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6a9aa-2faa-4d7d-a964-49224c9bc8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:00.050258 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6a9aa-2faa-4d7d-a964-49224c9bc8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:00.050522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6a9aa-2faa-4d7d-a964-49224c9bc8f9 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:00.051226 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6a9aa-2faa-4d7d-a964-49224c9bc8f9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:00.051538 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6a9aa-2faa-4d7d-a964-49224c9bc8f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:00.052978 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6a9aa-2faa-4d7d-a964-49224c9bc8f9 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:00.061025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 350/709] 10.222.0.61 () {56 vars in 1124 bytes} [Wed Sep 7 13:00:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:00.073365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6a9aa-2faa-4d7d-a964-49224c9bc8f9 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:00.151436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 360/710] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:01:00.151436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f02a8a-cd1e-407b-b306-f9eea188cdfb None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:00.179141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:00.275722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:00.326907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6a9aa-2faa-4d7d-a964-49224c9bc8f9 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:00.332519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f02a8a-cd1e-407b-b306-f9eea188cdfb None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:00.377653 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b4904-4089-45a8-9e19-701d1cbba604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:00.378703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b4904-4089-45a8-9e19-701d1cbba604 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:00.379436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b4904-4089-45a8-9e19-701d1cbba604 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:00.379928 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b4904-4089-45a8-9e19-701d1cbba604 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:00.388766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b4904-4089-45a8-9e19-701d1cbba604 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:00.391034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b4904-4089-45a8-9e19-701d1cbba604 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:00.400882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2758738-907e-401e-9d49-ed27017a3c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:00.401837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2758738-907e-401e-9d49-ed27017a3c02 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:00.402267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2758738-907e-401e-9d49-ed27017a3c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:00.402686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2758738-907e-401e-9d49-ed27017a3c02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-de7b4904-4089-45a8-9e19-701d1cbba604 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:01:00.410742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:00.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:00.412771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:01:00.412771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:01:00.412771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:01:00.412771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:01:00.412771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:01:00.412771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:01:00.412771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:00.412771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:00.414846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 361/711] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:00.425168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b4904-4089-45a8-9e19-701d1cbba604 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:00.426676 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2758738-907e-401e-9d49-ed27017a3c02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:00.428540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2758738-907e-401e-9d49-ed27017a3c02 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-c2758738-907e-401e-9d49-ed27017a3c02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:00.452405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:00.453433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:00.454324 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:00.454324 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:00.454324 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:00.462961 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 351/712] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:00.464363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2758738-907e-401e-9d49-ed27017a3c02 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:00.628114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b4904-4089-45a8-9e19-701d1cbba604 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:00.639563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:00.653681 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2758738-907e-401e-9d49-ed27017a3c02 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:00.673555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:00.700355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8111293-de19-4c18-b489-903d974c7214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:00.701192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8f634-3c25-47f6-8df0-f3a045d21f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:00.702438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8f634-3c25-47f6-8df0-f3a045d21f4d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:00.702648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8111293-de19-4c18-b489-903d974c7214 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:00.703311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8111293-de19-4c18-b489-903d974c7214 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:00.703694 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8f634-3c25-47f6-8df0-f3a045d21f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:00.703919 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8111293-de19-4c18-b489-903d974c7214 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:00.704463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8f634-3c25-47f6-8df0-f3a045d21f4d None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:00.705328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8f634-3c25-47f6-8df0-f3a045d21f4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:00.705821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8f634-3c25-47f6-8df0-f3a045d21f4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:00.707465 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8111293-de19-4c18-b489-903d974c7214 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:00.708060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8111293-de19-4c18-b489-903d974c7214 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:00.708615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8f634-3c25-47f6-8df0-f3a045d21f4d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:00.710965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8111293-de19-4c18-b489-903d974c7214 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:00.723752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 352/713] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:00.736492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8f634-3c25-47f6-8df0-f3a045d21f4d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:00.744563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 362/714] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:01:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:00.746420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8111293-de19-4c18-b489-903d974c7214 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:00.869736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8f634-3c25-47f6-8df0-f3a045d21f4d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:00.912045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002da102-e627-42ee-bd39-1220dc4193cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:00.913097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002da102-e627-42ee-bd39-1220dc4193cf None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:00.913588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002da102-e627-42ee-bd39-1220dc4193cf None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:00.914637 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002da102-e627-42ee-bd39-1220dc4193cf None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:00.915557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002da102-e627-42ee-bd39-1220dc4193cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:00.917398 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002da102-e627-42ee-bd39-1220dc4193cf None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:00.937990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:00.950683 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 353/715] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 13:01:00 2022] POST /identity/v3/users => generated 254 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:01:00.952494 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002da102-e627-42ee-bd39-1220dc4193cf None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:00.959669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8111293-de19-4c18-b489-903d974c7214 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:01.035853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f54cc-81ed-4cfa-b83c-e9bc54acf13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:01.036503 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f54cc-81ed-4cfa-b83c-e9bc54acf13e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:01.037062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f54cc-81ed-4cfa-b83c-e9bc54acf13e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:01.037568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f54cc-81ed-4cfa-b83c-e9bc54acf13e None admin] PATH_INFO: `/v3/services/network` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:01.038878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f54cc-81ed-4cfa-b83c-e9bc54acf13e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:01.042616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f54cc-81ed-4cfa-b83c-e9bc54acf13e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-029f54cc-81ed-4cfa-b83c-e9bc54acf13e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:01.064885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:01.066032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:01:01.066880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:01.070924 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 363/716] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:00 2022] GET /identity/v3/services/network => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:01.082272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f54cc-81ed-4cfa-b83c-e9bc54acf13e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:01.146195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002da102-e627-42ee-bd39-1220dc4193cf None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:01.189636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:01.238434 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56317e1-5e12-46ea-adad-12f3c7ce50d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:01.239163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56317e1-5e12-46ea-adad-12f3c7ce50d5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:01.239434 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56317e1-5e12-46ea-adad-12f3c7ce50d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:01.239701 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56317e1-5e12-46ea-adad-12f3c7ce50d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:01.240746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56317e1-5e12-46ea-adad-12f3c7ce50d5 None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=ba93e2d4aaed4c5595d5fd7363626bcb) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:01.241067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56317e1-5e12-46ea-adad-12f3c7ce50d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:01.243280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56317e1-5e12-46ea-adad-12f3c7ce50d5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:01.261297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 354/717] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:01:00 2022] GET /identity/v3/role_assignments?user.id=ba93e2d4aaed4c5595d5fd7363626bcb&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:01.300892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f54cc-81ed-4cfa-b83c-e9bc54acf13e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:01.358365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e8883-d798-467b-9b66-6aa8fe472630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:01.359069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e8883-d798-467b-9b66-6aa8fe472630 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:01.359379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e8883-d798-467b-9b66-6aa8fe472630 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:01.359655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e8883-d798-467b-9b66-6aa8fe472630 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:01.360324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e8883-d798-467b-9b66-6aa8fe472630 None admin] RBAC: Adding query filter params (name=network) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:01.360637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e8883-d798-467b-9b66-6aa8fe472630 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:01.362075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e8883-d798-467b-9b66-6aa8fe472630 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:01.368881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 364/718] 10.222.0.61 () {56 vars in 1112 bytes} [Wed Sep 7 13:01:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:01.378469 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56317e1-5e12-46ea-adad-12f3c7ce50d5 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:01.519265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:01.607869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56317e1-5e12-46ea-adad-12f3c7ce50d5 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:01.680383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30233f-44fc-4ad7-9616-60d5fa3ee829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:01.681386 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30233f-44fc-4ad7-9616-60d5fa3ee829 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:01.681860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30233f-44fc-4ad7-9616-60d5fa3ee829 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:01.682338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30233f-44fc-4ad7-9616-60d5fa3ee829 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:01.683526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30233f-44fc-4ad7-9616-60d5fa3ee829 None admin] RBAC: Adding query filter params (type=network) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:01.684056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30233f-44fc-4ad7-9616-60d5fa3ee829 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:01.686662 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30233f-44fc-4ad7-9616-60d5fa3ee829 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:01.696798 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 355/719] 10.222.0.61 () {56 vars in 1112 bytes} [Wed Sep 7 13:01:01 2022] GET /identity/v3/services?type=network => generated 124 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:03.501856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba9641-8aed-4f95-a403-e97f10eb4b39 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:03.503416 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba9641-8aed-4f95-a403-e97f10eb4b39 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:03.504239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba9641-8aed-4f95-a403-e97f10eb4b39 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:03.510636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 365/720] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:03.517630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae602922-8dcc-4855-80fe-5ee6453ef2ef None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:03.518603 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae602922-8dcc-4855-80fe-5ee6453ef2ef None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:03.519128 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae602922-8dcc-4855-80fe-5ee6453ef2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:03.548385 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ae602922-8dcc-4855-80fe-5ee6453ef2ef None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:03.550021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ae602922-8dcc-4855-80fe-5ee6453ef2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:03.643669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 356/721] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:01:04.122517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba9641-8aed-4f95-a403-e97f10eb4b39 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:04.180031 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50979c8-95dc-4991-b66b-747f4bd0f1b8 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:04.180777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50979c8-95dc-4991-b66b-747f4bd0f1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:04.181274 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50979c8-95dc-4991-b66b-747f4bd0f1b8 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:04.183652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 357/722] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:04.190576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae254fc-4686-4ab2-b970-9ecee7f50f32 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:04.191332 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae254fc-4686-4ab2-b970-9ecee7f50f32 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:04.191813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae254fc-4686-4ab2-b970-9ecee7f50f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:04.230742 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-5ae254fc-4686-4ab2-b970-9ecee7f50f32 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:04.232593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5ae254fc-4686-4ab2-b970-9ecee7f50f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:04.311809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:04.325646 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 358/723] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:01:04.369550 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba9641-8aed-4f95-a403-e97f10eb4b39 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:04.383830 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:04.418567 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b2987-60fb-4dc2-9aab-2867b597bfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:04.419292 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b2987-60fb-4dc2-9aab-2867b597bfad None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:04.419579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b2987-60fb-4dc2-9aab-2867b597bfad None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:04.419927 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b2987-60fb-4dc2-9aab-2867b597bfad None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:04.420790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b2987-60fb-4dc2-9aab-2867b597bfad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:04.422385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b2987-60fb-4dc2-9aab-2867b597bfad None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:04.448605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 366/724] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 13:01:04 2022] POST /identity/v3/services => generated 251 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:01:04.711425 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffe3ae-d19c-4798-b6f2-624f290dac1d None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:04.713037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffe3ae-d19c-4798-b6f2-624f290dac1d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:04.713842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffe3ae-d19c-4798-b6f2-624f290dac1d None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:04.717351 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 359/725] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:04.727664 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01361bbb-5c1d-4f52-8836-8c68d6719e3c None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:04.729221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01361bbb-5c1d-4f52-8836-8c68d6719e3c None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:04.729797 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01361bbb-5c1d-4f52-8836-8c68d6719e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:04.739294 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ffe3ae-d19c-4798-b6f2-624f290dac1d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:04.757466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-01361bbb-5c1d-4f52-8836-8c68d6719e3c None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:04.759200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-01361bbb-5c1d-4f52-8836-8c68d6719e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:04.894103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 367/726] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:01:04.896070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811f6a5-852c-43fd-8e83-97ad926ee874 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:04.896584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811f6a5-852c-43fd-8e83-97ad926ee874 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:04.897044 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811f6a5-852c-43fd-8e83-97ad926ee874 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:04.899590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 368/727] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:04.904858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e97357-6bc1-4b78-a8e7-a9e1469bb3bb None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:04.905793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e97357-6bc1-4b78-a8e7-a9e1469bb3bb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:04.906395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e97357-6bc1-4b78-a8e7-a9e1469bb3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:04.937390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-67e97357-6bc1-4b78-a8e7-a9e1469bb3bb None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:04.939924 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-67e97357-6bc1-4b78-a8e7-a9e1469bb3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:04.958459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ffe3ae-d19c-4798-b6f2-624f290dac1d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:05.044880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5563e-391d-4cc1-b2f8-1b341c65777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:05.045960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5563e-391d-4cc1-b2f8-1b341c65777a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:05.046504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5563e-391d-4cc1-b2f8-1b341c65777a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:05.047093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5563e-391d-4cc1-b2f8-1b341c65777a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:05.064076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 369/728] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:01:05.074784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5563e-391d-4cc1-b2f8-1b341c65777a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-3ab5563e-391d-4cc1-b2f8-1b341c65777a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:01:05.080460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:05.081729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:05.082925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:05.082925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:05.082925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:05.082925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:05.082925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:01:05.082925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:01:05.082925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:01:05.082925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:01:05.082925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:01:05.082925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:01:05.082925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:01:05.082925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:05.082925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 360/729] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:01:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:05.092591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e97357-6bc1-4b78-a8e7-a9e1469bb3bb None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:05.139534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:05.275382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e97357-6bc1-4b78-a8e7-a9e1469bb3bb None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:05.349732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc3e42-9ab2-4605-a9f5-12e04d3a598e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:05.350384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc3e42-9ab2-4605-a9f5-12e04d3a598e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:05.350685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc3e42-9ab2-4605-a9f5-12e04d3a598e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:05.351029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc3e42-9ab2-4605-a9f5-12e04d3a598e None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:05.352001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc3e42-9ab2-4605-a9f5-12e04d3a598e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:05.352309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc3e42-9ab2-4605-a9f5-12e04d3a598e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:05.353736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc3e42-9ab2-4605-a9f5-12e04d3a598e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:05.369346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 370/730] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:01:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:05.378872 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5563e-391d-4cc1-b2f8-1b341c65777a None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:05.432808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc3e42-9ab2-4605-a9f5-12e04d3a598e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:05.486031 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:05.563015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5563e-391d-4cc1-b2f8-1b341c65777a None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:05.582903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:05.603042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc3e42-9ab2-4605-a9f5-12e04d3a598e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:05.625173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa612e-6ce6-4911-b703-b43c36915340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:05.626493 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa612e-6ce6-4911-b703-b43c36915340 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:05.626990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa612e-6ce6-4911-b703-b43c36915340 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:05.627510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa612e-6ce6-4911-b703-b43c36915340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:05.640785 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa612e-6ce6-4911-b703-b43c36915340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:05.642278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa612e-6ce6-4911-b703-b43c36915340 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:05.646985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577da06-d072-49ed-90d5-d04c55919f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:05.648677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577da06-d072-49ed-90d5-d04c55919f47 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:05.649177 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577da06-d072-49ed-90d5-d04c55919f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:05.649680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577da06-d072-49ed-90d5-d04c55919f47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-5dfa612e-6ce6-4911-b703-b43c36915340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:05.660507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:05.661562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:05.662492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:05.662492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:05.662492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:05.662492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 361/731] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:05.663222 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b7f86-8518-4d44-b4e1-ddf2ad02c221 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:05.663479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b7f86-8518-4d44-b4e1-ddf2ad02c221 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:05.664292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b7f86-8518-4d44-b4e1-ddf2ad02c221 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:05.670002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 362/732] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:05.670002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b7f86-8518-4d44-b4e1-ddf2ad02c221 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:05.676377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577da06-d072-49ed-90d5-d04c55919f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-5577da06-d072-49ed-90d5-d04c55919f47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:01:05.686197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:05.687356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:05.688237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:05.688237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:05.688237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:05.688237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:05.688237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:01:05.688237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:01:05.688237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:01:05.688237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:01:05.688237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:01:05.688237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:01:05.688237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:01:05.688237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:05.695622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 371/733] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:01:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:05.697704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b985b6f-1531-473d-b323-7c308ac6db62 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:05.698148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b985b6f-1531-473d-b323-7c308ac6db62 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:05.698618 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b985b6f-1531-473d-b323-7c308ac6db62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:05.730409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-5b985b6f-1531-473d-b323-7c308ac6db62 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:05.732415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5b985b6f-1531-473d-b323-7c308ac6db62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:05.810902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:05.822372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b7f86-8518-4d44-b4e1-ddf2ad02c221 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:05.838385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 372/734] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:01:05.839322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b985b6f-1531-473d-b323-7c308ac6db62 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:05.883221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd77bf-e7e2-4532-a966-f2259da69c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:05.884262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd77bf-e7e2-4532-a966-f2259da69c76 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:05.884805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd77bf-e7e2-4532-a966-f2259da69c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:05.885312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd77bf-e7e2-4532-a966-f2259da69c76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:05.901787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd77bf-e7e2-4532-a966-f2259da69c76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-38cd77bf-e7e2-4532-a966-f2259da69c76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:01:05.907043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:05.908082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:05.908924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:05.908924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:05.908924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:05.908924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:05.908924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:01:05.908924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:01:05.908924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:01:05.908924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:01:05.908924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:01:05.908924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:01:05.908924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:01:05.908924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:05.908924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 363/735] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:01:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:05.909609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd77bf-e7e2-4532-a966-f2259da69c76 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:05.987647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:06.002454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b985b6f-1531-473d-b323-7c308ac6db62 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:06.068286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12669b4-0575-4281-aba6-be3ec707303d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:06.069189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12669b4-0575-4281-aba6-be3ec707303d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:06.069742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12669b4-0575-4281-aba6-be3ec707303d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:06.070522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12669b4-0575-4281-aba6-be3ec707303d None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:06.071965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12669b4-0575-4281-aba6-be3ec707303d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:06.072745 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12669b4-0575-4281-aba6-be3ec707303d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:06.076839 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12669b4-0575-4281-aba6-be3ec707303d None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:06.098154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 373/736] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:06.099186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12669b4-0575-4281-aba6-be3ec707303d None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:06.108787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd77bf-e7e2-4532-a966-f2259da69c76 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:06.156588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59eed4-508c-4517-849a-a600e38df40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:06.157043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59eed4-508c-4517-849a-a600e38df40c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:06.157362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59eed4-508c-4517-849a-a600e38df40c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:06.157621 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59eed4-508c-4517-849a-a600e38df40c None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:06.158304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59eed4-508c-4517-849a-a600e38df40c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:06.158612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59eed4-508c-4517-849a-a600e38df40c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:06.160166 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59eed4-508c-4517-849a-a600e38df40c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:06.171891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 364/737] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:01:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:06.172864 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59eed4-508c-4517-849a-a600e38df40c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:06.202011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:06.286469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12669b4-0575-4281-aba6-be3ec707303d None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:06.312818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59eed4-508c-4517-849a-a600e38df40c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:06.334531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8af641-babb-4165-9be4-4b4b1910459c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:06.336218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8af641-babb-4165-9be4-4b4b1910459c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:06.336615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8af641-babb-4165-9be4-4b4b1910459c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:06.337034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8af641-babb-4165-9be4-4b4b1910459c None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:06.338157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8af641-babb-4165-9be4-4b4b1910459c None admin] RBAC: Adding query filter params (name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:06.338657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8af641-babb-4165-9be4-4b4b1910459c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:06.340999 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8af641-babb-4165-9be4-4b4b1910459c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:06.350221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:06.361875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 374/738] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:01:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:06.361875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8af641-babb-4165-9be4-4b4b1910459c None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:06.377315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fe850-be4d-4480-9693-ae1f870389c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:06.378314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fe850-be4d-4480-9693-ae1f870389c4 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:06.378763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fe850-be4d-4480-9693-ae1f870389c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:06.379254 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fe850-be4d-4480-9693-ae1f870389c4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:06.393999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fe850-be4d-4480-9693-ae1f870389c4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:06.396600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fe850-be4d-4480-9693-ae1f870389c4 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-373fe850-be4d-4480-9693-ae1f870389c4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:01:06.419969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:06.421184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:06.422600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:06.422600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:06.422600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:01:06.422600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:01:06.422600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:01:06.422600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:01:06.422600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:01:06.422600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:01:06.422600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:01:06.422600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:06.422600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 365/739] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:01:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:06.423290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fe850-be4d-4480-9693-ae1f870389c4 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:06.520220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8af641-babb-4165-9be4-4b4b1910459c None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:06.560068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:06.576145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163e537-9a57-4753-8b9f-1a2320b318a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:06.577478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163e537-9a57-4753-8b9f-1a2320b318a2 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:06.577934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163e537-9a57-4753-8b9f-1a2320b318a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:06.578410 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163e537-9a57-4753-8b9f-1a2320b318a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:06.591517 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fe850-be4d-4480-9693-ae1f870389c4 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:06.600604 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163e537-9a57-4753-8b9f-1a2320b318a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:06.603472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163e537-9a57-4753-8b9f-1a2320b318a2 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-e163e537-9a57-4753-8b9f-1a2320b318a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:06.633671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:06.634878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:06.636029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:06.636029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:06.636029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:06.636029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 375/740] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:06.637216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163e537-9a57-4753-8b9f-1a2320b318a2 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:06.648616 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634568b-cf82-4cc8-8819-adcfee25cb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:06.649476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634568b-cf82-4cc8-8819-adcfee25cb44 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:06.649815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634568b-cf82-4cc8-8819-adcfee25cb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:06.650150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634568b-cf82-4cc8-8819-adcfee25cb44 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:06.650927 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634568b-cf82-4cc8-8819-adcfee25cb44 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:06.653408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634568b-cf82-4cc8-8819-adcfee25cb44 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:06.676924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 366/741] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 13:01:06 2022] POST /identity/v3/services => generated 240 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:01:06.678455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634568b-cf82-4cc8-8819-adcfee25cb44 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:06.769051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:06.907871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163e537-9a57-4753-8b9f-1a2320b318a2 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:06.957864 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634568b-cf82-4cc8-8819-adcfee25cb44 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:06.973518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133af33-1c5c-4f60-bfaf-f112df04a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:06.974199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133af33-1c5c-4f60-bfaf-f112df04a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:06.974484 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133af33-1c5c-4f60-bfaf-f112df04a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:06.974759 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133af33-1c5c-4f60-bfaf-f112df04a5e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:06.996600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133af33-1c5c-4f60-bfaf-f112df04a5e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:07.000149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133af33-1c5c-4f60-bfaf-f112df04a5e9 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:07.030312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7599b38-adc4-4286-9fb6-6649b0f225c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:07.031375 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7599b38-adc4-4286-9fb6-6649b0f225c0 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:07.031840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7599b38-adc4-4286-9fb6-6649b0f225c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:07.032312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7599b38-adc4-4286-9fb6-6649b0f225c0 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:07.033717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7599b38-adc4-4286-9fb6-6649b0f225c0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:07.034223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7599b38-adc4-4286-9fb6-6649b0f225c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-b133af33-1c5c-4f60-bfaf-f112df04a5e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:07.036227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:07.038063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:07.040095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:07.040095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:07.040095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:07.040427 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7599b38-adc4-4286-9fb6-6649b0f225c0 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:07.042221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:07.048042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 376/742] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:07.050126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133af33-1c5c-4f60-bfaf-f112df04a5e9 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:07.077580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 367/743] 10.222.0.61 () {56 vars in 1135 bytes} [Wed Sep 7 13:01:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:07.079415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7599b38-adc4-4286-9fb6-6649b0f225c0 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:07.231903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133af33-1c5c-4f60-bfaf-f112df04a5e9 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:07.235269 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7599b38-adc4-4286-9fb6-6649b0f225c0 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:07.236879 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:07.288288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d976b3-3a97-4864-a2dc-39f9bf32e66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:07.289464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d976b3-3a97-4864-a2dc-39f9bf32e66b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:07.290077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d976b3-3a97-4864-a2dc-39f9bf32e66b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:07.290360 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d976b3-3a97-4864-a2dc-39f9bf32e66b None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:07.291039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d976b3-3a97-4864-a2dc-39f9bf32e66b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:07.291992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d976b3-3a97-4864-a2dc-39f9bf32e66b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:07.293827 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d976b3-3a97-4864-a2dc-39f9bf32e66b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:07.307424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ff2f6-19c3-4d8d-b78b-11cc09cb035b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:07.308495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ff2f6-19c3-4d8d-b78b-11cc09cb035b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:07.308966 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ff2f6-19c3-4d8d-b78b-11cc09cb035b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:07.309420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ff2f6-19c3-4d8d-b78b-11cc09cb035b None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:07.310611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ff2f6-19c3-4d8d-b78b-11cc09cb035b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:07.311160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ff2f6-19c3-4d8d-b78b-11cc09cb035b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:07.313949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ff2f6-19c3-4d8d-b78b-11cc09cb035b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:07.315462 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 368/744] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:07.316526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d976b3-3a97-4864-a2dc-39f9bf32e66b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:07.339583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 377/745] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:07.341366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ff2f6-19c3-4d8d-b78b-11cc09cb035b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:07.396032 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:07.467091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d976b3-3a97-4864-a2dc-39f9bf32e66b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:07.519085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e001425-fc97-47ea-8964-799919364d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:07.519654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e001425-fc97-47ea-8964-799919364d7d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:07.519934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e001425-fc97-47ea-8964-799919364d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:07.520209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e001425-fc97-47ea-8964-799919364d7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:07.548450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ff2f6-19c3-4d8d-b78b-11cc09cb035b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:07.614871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d033d-2835-48c7-be1e-3da53204248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:07.615844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d033d-2835-48c7-be1e-3da53204248d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:07.616128 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d033d-2835-48c7-be1e-3da53204248d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:07.616412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d033d-2835-48c7-be1e-3da53204248d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:07.628215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d033d-2835-48c7-be1e-3da53204248d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:07.630395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d033d-2835-48c7-be1e-3da53204248d None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:07.642667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e001425-fc97-47ea-8964-799919364d7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-618d033d-2835-48c7-be1e-3da53204248d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:01:07.645516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:07.647988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:07.651504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:07.651504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:07.651504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:01:07.651504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:01:07.651504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:01:07.651504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:01:07.651504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:01:07.651504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:01:07.651504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:01:07.651504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:07.653283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e001425-fc97-47ea-8964-799919364d7d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:07.653437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 378/746] 10.222.0.61 () {56 vars in 1117 bytes} [Wed Sep 7 13:01:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:07.653437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d033d-2835-48c7-be1e-3da53204248d None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:07.689065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:07.770102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:07.808318 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d033d-2835-48c7-be1e-3da53204248d None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:07.850421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 369/747] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:01:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:01:07.852147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e001425-fc97-47ea-8964-799919364d7d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:07.893867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476123b-f0d5-4b35-bf01-adb515791fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:07.895710 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476123b-f0d5-4b35-bf01-adb515791fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:07.896225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476123b-f0d5-4b35-bf01-adb515791fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:07.896733 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476123b-f0d5-4b35-bf01-adb515791fc7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:07.917764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476123b-f0d5-4b35-bf01-adb515791fc7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:07.922199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476123b-f0d5-4b35-bf01-adb515791fc7 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-d476123b-f0d5-4b35-bf01-adb515791fc7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:01:07.949778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:07.951996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:07.956042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:07.956042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:07.956042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:01:07.956042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:01:07.956042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:01:07.956042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:01:07.956042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:01:07.956042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:01:07.956042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:01:07.956042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:07.956042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 379/748] 10.222.0.61 () {56 vars in 1093 bytes} [Wed Sep 7 13:01:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:07.956042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476123b-f0d5-4b35-bf01-adb515791fc7 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:08.114036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e001425-fc97-47ea-8964-799919364d7d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:08.156466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:08.197707 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4e459-5e58-4bd7-a7a6-e459efd81037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:08.198823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4e459-5e58-4bd7-a7a6-e459efd81037 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:08.199385 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4e459-5e58-4bd7-a7a6-e459efd81037 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:08.199902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4e459-5e58-4bd7-a7a6-e459efd81037 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:08.200509 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476123b-f0d5-4b35-bf01-adb515791fc7 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:08.201218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4e459-5e58-4bd7-a7a6-e459efd81037 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:08.201758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4e459-5e58-4bd7-a7a6-e459efd81037 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:08.203782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4e459-5e58-4bd7-a7a6-e459efd81037 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:08.228377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 370/749] 10.222.0.61 () {56 vars in 1149 bytes} [Wed Sep 7 13:01:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:08.229697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4e459-5e58-4bd7-a7a6-e459efd81037 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:08.256934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:08.272974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab659a4f-b5bf-4ecc-bad7-5b973c4e9da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:08.273907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab659a4f-b5bf-4ecc-bad7-5b973c4e9da3 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:08.274386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab659a4f-b5bf-4ecc-bad7-5b973c4e9da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:08.274811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab659a4f-b5bf-4ecc-bad7-5b973c4e9da3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:08.294173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab659a4f-b5bf-4ecc-bad7-5b973c4e9da3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:08.297174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab659a4f-b5bf-4ecc-bad7-5b973c4e9da3 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-ab659a4f-b5bf-4ecc-bad7-5b973c4e9da3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:08.322492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:08.324588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:08.326055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:08.326055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:08.326055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:08.326317 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 380/750] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:08.328565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258564ea-cc77-43e6-b24a-352e28bc85a2 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:08.329059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258564ea-cc77-43e6-b24a-352e28bc85a2 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:08.329554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258564ea-cc77-43e6-b24a-352e28bc85a2 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:08.332376 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 381/751] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:08.337158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258564ea-cc77-43e6-b24a-352e28bc85a2 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:08.412180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4e459-5e58-4bd7-a7a6-e459efd81037 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:08.475322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a5509-a1b6-4a4c-8839-6f8f676eece7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:08.476990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a5509-a1b6-4a4c-8839-6f8f676eece7 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:08.477489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a5509-a1b6-4a4c-8839-6f8f676eece7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:08.477967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a5509-a1b6-4a4c-8839-6f8f676eece7 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:08.479398 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a5509-a1b6-4a4c-8839-6f8f676eece7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:08.479947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a5509-a1b6-4a4c-8839-6f8f676eece7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:08.482712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a5509-a1b6-4a4c-8839-6f8f676eece7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:08.519536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 371/752] 10.222.0.61 () {56 vars in 1133 bytes} [Wed Sep 7 13:01:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:08.519536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a5509-a1b6-4a4c-8839-6f8f676eece7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:08.522692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258564ea-cc77-43e6-b24a-352e28bc85a2 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:08.571149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11dbef-a2e4-411f-aba0-e7dd27680260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:08.572195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11dbef-a2e4-411f-aba0-e7dd27680260 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:08.572621 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11dbef-a2e4-411f-aba0-e7dd27680260 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:08.573034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11dbef-a2e4-411f-aba0-e7dd27680260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:08.578084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:08.655807 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:08.674148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11dbef-a2e4-411f-aba0-e7dd27680260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:08.676125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11dbef-a2e4-411f-aba0-e7dd27680260 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:08.719609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a5509-a1b6-4a4c-8839-6f8f676eece7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:08.790997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea6447-bc26-446c-a84a-630097fa5c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:08.792200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6447-bc26-446c-a84a-630097fa5c46 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:08.792657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6447-bc26-446c-a84a-630097fa5c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:08.793066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6447-bc26-446c-a84a-630097fa5c46 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:08.794180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea6447-bc26-446c-a84a-630097fa5c46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:08.794712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea6447-bc26-446c-a84a-630097fa5c46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:08.797598 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea6447-bc26-446c-a84a-630097fa5c46 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:08.811923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 382/753] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:01:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:01:08.811923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7889c06-6efc-44be-a711-870f495d3825 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:08.811923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7889c06-6efc-44be-a711-870f495d3825 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:08.811923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7889c06-6efc-44be-a711-870f495d3825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:08.828424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 372/754] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:08.829712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea6447-bc26-446c-a84a-630097fa5c46 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:08.841666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-d7889c06-6efc-44be-a711-870f495d3825 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:08.843939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d7889c06-6efc-44be-a711-870f495d3825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:08.904738 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:08.981480 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 383/755] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:08.983162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7889c06-6efc-44be-a711-870f495d3825 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:09.024309 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea6447-bc26-446c-a84a-630097fa5c46 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:09.092918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:09.102136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787aa7a-9d73-4c8d-a98a-28f5c476d6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:09.103312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787aa7a-9d73-4c8d-a98a-28f5c476d6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:09.103867 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787aa7a-9d73-4c8d-a98a-28f5c476d6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:09.104353 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787aa7a-9d73-4c8d-a98a-28f5c476d6e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:09.120180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787aa7a-9d73-4c8d-a98a-28f5c476d6e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:09.122568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787aa7a-9d73-4c8d-a98a-28f5c476d6e5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-2787aa7a-9d73-4c8d-a98a-28f5c476d6e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:09.143570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:09.144613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:09.145495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:09.145495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:09.145495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:09.148826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 373/756] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:09.149923 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787aa7a-9d73-4c8d-a98a-28f5c476d6e5 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:09.207219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7889c06-6efc-44be-a711-870f495d3825 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:09.280842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81288f-51e8-4274-9947-6294181c3e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:09.282069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81288f-51e8-4274-9947-6294181c3e67 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:09.282688 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81288f-51e8-4274-9947-6294181c3e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:09.283335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81288f-51e8-4274-9947-6294181c3e67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:09.293656 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787aa7a-9d73-4c8d-a98a-28f5c476d6e5 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:09.306946 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81288f-51e8-4274-9947-6294181c3e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:09.310094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81288f-51e8-4274-9947-6294181c3e67 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-bd81288f-51e8-4274-9947-6294181c3e67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:09.344092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:09.345374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:09.346337 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:09.346337 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:09.346337 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:09.347392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 384/757] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:09.348302 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e6558-152e-4dcc-8336-74880af451fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:09.349185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e6558-152e-4dcc-8336-74880af451fd None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:09.349320 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81288f-51e8-4274-9947-6294181c3e67 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:09.349803 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e6558-152e-4dcc-8336-74880af451fd None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:09.350379 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e6558-152e-4dcc-8336-74880af451fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:09.360716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e6558-152e-4dcc-8336-74880af451fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:09.362548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e6558-152e-4dcc-8336-74880af451fd None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-cd3e6558-152e-4dcc-8336-74880af451fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:01:09.378235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:09.379276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:09.380181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:01:09.380181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:01:09.380181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:01:09.380181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:01:09.380181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:01:09.380181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:01:09.380181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:09.380181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:09.382885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 374/758] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:09.384045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e6558-152e-4dcc-8336-74880af451fd None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:09.399928 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:09.451714 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:09.539973 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e6558-152e-4dcc-8336-74880af451fd None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:09.566302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81288f-51e8-4274-9947-6294181c3e67 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:09.587511 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00a641-009c-4f3f-ab75-c4e2d6ecde60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:09.588140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a641-009c-4f3f-ab75-c4e2d6ecde60 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:09.588423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a641-009c-4f3f-ab75-c4e2d6ecde60 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:09.588680 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a641-009c-4f3f-ab75-c4e2d6ecde60 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:09.589348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00a641-009c-4f3f-ab75-c4e2d6ecde60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:09.589640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00a641-009c-4f3f-ab75-c4e2d6ecde60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:09.591074 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00a641-009c-4f3f-ab75-c4e2d6ecde60 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:09.606882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 375/759] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:09.607472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00a641-009c-4f3f-ab75-c4e2d6ecde60 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:09.637759 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909b6a4-e909-40a4-9c0a-5cbc4b3404a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:09.638427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909b6a4-e909-40a4-9c0a-5cbc4b3404a1 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:09.638725 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909b6a4-e909-40a4-9c0a-5cbc4b3404a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:09.639079 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909b6a4-e909-40a4-9c0a-5cbc4b3404a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:09.639792 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909b6a4-e909-40a4-9c0a-5cbc4b3404a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:09.640097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909b6a4-e909-40a4-9c0a-5cbc4b3404a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:09.641513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909b6a4-e909-40a4-9c0a-5cbc4b3404a1 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:09.664055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 385/760] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:09.664055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909b6a4-e909-40a4-9c0a-5cbc4b3404a1 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:09.797535 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:09.802620 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00a641-009c-4f3f-ab75-c4e2d6ecde60 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:09.805861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:09.830300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909b6a4-e909-40a4-9c0a-5cbc4b3404a1 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:09.851292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3715400-9c50-47fe-ba5d-45a117a374c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:09.852008 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3715400-9c50-47fe-ba5d-45a117a374c0 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:09.852352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3715400-9c50-47fe-ba5d-45a117a374c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:09.852695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3715400-9c50-47fe-ba5d-45a117a374c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:09.853491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3715400-9c50-47fe-ba5d-45a117a374c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:09.853876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3715400-9c50-47fe-ba5d-45a117a374c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:09.855560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3715400-9c50-47fe-ba5d-45a117a374c0 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:09.871192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 376/761] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:01:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:09.871436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21010af6-dac3-4740-97f1-2564726c43b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:09.871902 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21010af6-dac3-4740-97f1-2564726c43b6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:09.872182 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21010af6-dac3-4740-97f1-2564726c43b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:09.872453 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21010af6-dac3-4740-97f1-2564726c43b6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:09.872618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3715400-9c50-47fe-ba5d-45a117a374c0 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:09.873107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21010af6-dac3-4740-97f1-2564726c43b6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:09.874451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21010af6-dac3-4740-97f1-2564726c43b6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-21010af6-dac3-4740-97f1-2564726c43b6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:09.888586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:09.890557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:09.892242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 386/762] 10.222.0.61 () {56 vars in 1129 bytes} [Wed Sep 7 13:01:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:09.893808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21010af6-dac3-4740-97f1-2564726c43b6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:10.027619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3715400-9c50-47fe-ba5d-45a117a374c0 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:10.043253 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21010af6-dac3-4740-97f1-2564726c43b6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:10.074389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52f54e-8975-467f-ba0d-e5b2b1109782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:10.075439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52f54e-8975-467f-ba0d-e5b2b1109782 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:10.075902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52f54e-8975-467f-ba0d-e5b2b1109782 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:10.076339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52f54e-8975-467f-ba0d-e5b2b1109782 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:10.088497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52f54e-8975-467f-ba0d-e5b2b1109782 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:10.091052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52f54e-8975-467f-ba0d-e5b2b1109782 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:10.100116 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a2453-c1f9-4d0d-b0a9-c5e0e7635909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:10.101149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a2453-c1f9-4d0d-b0a9-c5e0e7635909 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:10.101614 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a2453-c1f9-4d0d-b0a9-c5e0e7635909 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:10.102078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a2453-c1f9-4d0d-b0a9-c5e0e7635909 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-cb52f54e-8975-467f-ba0d-e5b2b1109782 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:01:10.112529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:10.113729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:10.114998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:01:10.114998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:01:10.114998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:01:10.114998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:01:10.114998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:01:10.114998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:01:10.114998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:10.114998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:10.114998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 377/763] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:10.117502 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a2453-c1f9-4d0d-b0a9-c5e0e7635909 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:10.117695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d6154-20bb-4c42-a00e-e6d9929be775 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:10.117695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d6154-20bb-4c42-a00e-e6d9929be775 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:10.118065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a2453-c1f9-4d0d-b0a9-c5e0e7635909 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:10.118249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d6154-20bb-4c42-a00e-e6d9929be775 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:10.119463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 378/764] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:10.120388 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d6154-20bb-4c42-a00e-e6d9929be775 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-445a2453-c1f9-4d0d-b0a9-c5e0e7635909 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:01:10.143900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:10.145857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:10.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:01:10.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:01:10.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:01:10.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:01:10.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:01:10.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:01:10.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:10.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:10.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 387/765] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:10.148565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a2453-c1f9-4d0d-b0a9-c5e0e7635909 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:10.156626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:10.163506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:10.273989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d6154-20bb-4c42-a00e-e6d9929be775 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:10.294898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a2453-c1f9-4d0d-b0a9-c5e0e7635909 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:10.323546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a7e64-dfad-49ac-9ca2-c3cdbb32c8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:10.324474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a7e64-dfad-49ac-9ca2-c3cdbb32c8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:10.324934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a7e64-dfad-49ac-9ca2-c3cdbb32c8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:10.325382 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a7e64-dfad-49ac-9ca2-c3cdbb32c8f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:10.326296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a7e64-dfad-49ac-9ca2-c3cdbb32c8f1 None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=0f886e0999d34bf9870b448772cc38a3) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:10.326796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a7e64-dfad-49ac-9ca2-c3cdbb32c8f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:10.329309 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a7e64-dfad-49ac-9ca2-c3cdbb32c8f1 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:10.349185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 379/766] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:01:10 2022] GET /identity/v3/role_assignments?user.id=0f886e0999d34bf9870b448772cc38a3&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:10.349185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a7e64-dfad-49ac-9ca2-c3cdbb32c8f1 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:10.349594 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aee0b2-469d-452b-b2b4-25fda871eb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:10.349594 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aee0b2-469d-452b-b2b4-25fda871eb4d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:10.349594 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aee0b2-469d-452b-b2b4-25fda871eb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:10.349594 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aee0b2-469d-452b-b2b4-25fda871eb4d None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:10.350053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aee0b2-469d-452b-b2b4-25fda871eb4d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:10.350053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aee0b2-469d-452b-b2b4-25fda871eb4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:10.350053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aee0b2-469d-452b-b2b4-25fda871eb4d None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:10.357888 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 388/767] 10.222.0.61 () {56 vars in 1124 bytes} [Wed Sep 7 13:01:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:10.360404 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850453c-c826-48e5-87eb-38fcc67353c7 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:10.361033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850453c-c826-48e5-87eb-38fcc67353c7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:10.361706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850453c-c826-48e5-87eb-38fcc67353c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:10.389603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-2850453c-c826-48e5-87eb-38fcc67353c7 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:10.392510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2850453c-c826-48e5-87eb-38fcc67353c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:10.507896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:10.520083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:10.532178 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 389/768] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:10.532178 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850453c-c826-48e5-87eb-38fcc67353c7 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:10.548372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a7e64-dfad-49ac-9ca2-c3cdbb32c8f1 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:10.618196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2322a-123e-4e2e-b161-e06d19d51370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:10.619291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2322a-123e-4e2e-b161-e06d19d51370 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:10.619869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2322a-123e-4e2e-b161-e06d19d51370 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:10.620492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2322a-123e-4e2e-b161-e06d19d51370 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:10.621845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2322a-123e-4e2e-b161-e06d19d51370 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:10.626577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2322a-123e-4e2e-b161-e06d19d51370 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:10.626577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2322a-123e-4e2e-b161-e06d19d51370 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:10.654812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 380/769] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:01:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:10.656592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2322a-123e-4e2e-b161-e06d19d51370 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:10.743017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850453c-c826-48e5-87eb-38fcc67353c7 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:10.816550 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1223bb4-3e59-4aae-87aa-581b82b9408b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:10.817619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1223bb4-3e59-4aae-87aa-581b82b9408b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:10.818112 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1223bb4-3e59-4aae-87aa-581b82b9408b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:10.818609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1223bb4-3e59-4aae-87aa-581b82b9408b None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:10.820614 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1223bb4-3e59-4aae-87aa-581b82b9408b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:10.821152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1223bb4-3e59-4aae-87aa-581b82b9408b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:10.824047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1223bb4-3e59-4aae-87aa-581b82b9408b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:10.855320 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 390/770] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:01:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:10.857026 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1223bb4-3e59-4aae-87aa-581b82b9408b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:10.973613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:11.065736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1223bb4-3e59-4aae-87aa-581b82b9408b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:11.068577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2322a-123e-4e2e-b161-e06d19d51370 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:11.123820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25e116-9f8d-43dd-80c0-e87a7bfb628a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:11.125040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25e116-9f8d-43dd-80c0-e87a7bfb628a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:11.125771 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25e116-9f8d-43dd-80c0-e87a7bfb628a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:11.126184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25e116-9f8d-43dd-80c0-e87a7bfb628a None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:11.127563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25e116-9f8d-43dd-80c0-e87a7bfb628a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:11.128049 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25e116-9f8d-43dd-80c0-e87a7bfb628a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:11.129838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25e116-9f8d-43dd-80c0-e87a7bfb628a None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:11.142457 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 381/771] 10.222.0.61 () {56 vars in 1124 bytes} [Wed Sep 7 13:01:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:11.142457 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25e116-9f8d-43dd-80c0-e87a7bfb628a None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:11.149221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a158a33-930d-4e0d-9fcb-d0b6ddd43d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:11.150303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a158a33-930d-4e0d-9fcb-d0b6ddd43d5e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:11.150788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a158a33-930d-4e0d-9fcb-d0b6ddd43d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:11.151301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a158a33-930d-4e0d-9fcb-d0b6ddd43d5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:11.152711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a158a33-930d-4e0d-9fcb-d0b6ddd43d5e None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=d71863f18330414f9b6238b7e8903701) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:11.153273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a158a33-930d-4e0d-9fcb-d0b6ddd43d5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:11.157030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a158a33-930d-4e0d-9fcb-d0b6ddd43d5e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:11.160063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:11.171683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 391/772] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:01:10 2022] GET /identity/v3/role_assignments?user.id=d71863f18330414f9b6238b7e8903701&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 259 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:11.173210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a158a33-930d-4e0d-9fcb-d0b6ddd43d5e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:11.380649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a158a33-930d-4e0d-9fcb-d0b6ddd43d5e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:11.392097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25e116-9f8d-43dd-80c0-e87a7bfb628a None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:11.424598 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:11.448391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3b33c-e181-470e-abd1-1b0324d6cf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:11.449068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3b33c-e181-470e-abd1-1b0324d6cf62 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:11.449464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3b33c-e181-470e-abd1-1b0324d6cf62 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:11.449747 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3b33c-e181-470e-abd1-1b0324d6cf62 None admin] PATH_INFO: `/v3/services/network` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:11.450478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3b33c-e181-470e-abd1-1b0324d6cf62 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:11.452278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3b33c-e181-470e-abd1-1b0324d6cf62 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-96a3b33c-e181-470e-abd1-1b0324d6cf62 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:11.463577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:11.466069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:11.469065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 392/773] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:11 2022] GET /identity/v3/services/network => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:11.471968 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3b33c-e181-470e-abd1-1b0324d6cf62 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:11.483159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a61cc-5f2b-4b5a-b7b0-7365e3c95c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:11.483159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a61cc-5f2b-4b5a-b7b0-7365e3c95c11 None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:11.483835 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a61cc-5f2b-4b5a-b7b0-7365e3c95c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:11.484053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a61cc-5f2b-4b5a-b7b0-7365e3c95c11 None admin] PATH_INFO: `/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/ba93e2d4aaed4c5595d5fd7363626bcb/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:11.488341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a61cc-5f2b-4b5a-b7b0-7365e3c95c11 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user_id=ba93e2d4aaed4c5595d5fd7363626bcb, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:11.491320 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a61cc-5f2b-4b5a-b7b0-7365e3c95c11 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:11.519846 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 382/774] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:01:11 2022] PUT /identity/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/ba93e2d4aaed4c5595d5fd7363626bcb/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:01:11.521324 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a61cc-5f2b-4b5a-b7b0-7365e3c95c11 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:11.698865 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3b33c-e181-470e-abd1-1b0324d6cf62 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:11.736659 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:11.799391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de266a6-178e-4e4e-ac56-efc50dd89587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:11.800049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de266a6-178e-4e4e-ac56-efc50dd89587 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:11.800326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de266a6-178e-4e4e-ac56-efc50dd89587 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:11.800625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de266a6-178e-4e4e-ac56-efc50dd89587 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:11.801366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de266a6-178e-4e4e-ac56-efc50dd89587 None admin] RBAC: Adding query filter params (interface=public, service_id=a9a69894ee2149e1a9d3ca2a3f3f4192, region_id=RegionOne) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:11.801684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de266a6-178e-4e4e-ac56-efc50dd89587 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:11.803514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de266a6-178e-4e4e-ac56-efc50dd89587 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:11.819146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 393/775] 10.222.0.61 () {56 vars in 1251 bytes} [Wed Sep 7 13:01:11 2022] GET /identity/v3/endpoints?service_id=a9a69894ee2149e1a9d3ca2a3f3f4192&interface=public®ion_id=RegionOne => generated 194 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:11.822977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a61cc-5f2b-4b5a-b7b0-7365e3c95c11 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:11.833701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de266a6-178e-4e4e-ac56-efc50dd89587 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:11.901779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc807a7-4d37-4196-a1af-9986417db53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:11.902440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc807a7-4d37-4196-a1af-9986417db53e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:11.902718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc807a7-4d37-4196-a1af-9986417db53e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:11.902988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc807a7-4d37-4196-a1af-9986417db53e None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:11.904115 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc807a7-4d37-4196-a1af-9986417db53e None admin] RBAC: Adding query filter params (name=network) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:11.904115 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc807a7-4d37-4196-a1af-9986417db53e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:11.905486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc807a7-4d37-4196-a1af-9986417db53e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:11.910964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 383/776] 10.222.0.61 () {56 vars in 1112 bytes} [Wed Sep 7 13:01:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:11.920933 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc807a7-4d37-4196-a1af-9986417db53e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:11.951361 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:12.115495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de266a6-178e-4e4e-ac56-efc50dd89587 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:12.185241 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa8f59-fb5f-4463-8cd4-8074f204c346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:12.186329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa8f59-fb5f-4463-8cd4-8074f204c346 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:12.186883 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa8f59-fb5f-4463-8cd4-8074f204c346 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:12.187402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa8f59-fb5f-4463-8cd4-8074f204c346 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:12.188468 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa8f59-fb5f-4463-8cd4-8074f204c346 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:12.191207 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa8f59-fb5f-4463-8cd4-8074f204c346 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:12.207476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 394/777] 10.222.0.61 () {56 vars in 1087 bytes} [Wed Sep 7 13:01:11 2022] GET /identity/v3/services => generated 772 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:12.267964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc807a7-4d37-4196-a1af-9986417db53e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:12.348118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231fe77-7e91-4adf-a503-3d8ced406143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:12.349176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231fe77-7e91-4adf-a503-3d8ced406143 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:12.349662 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231fe77-7e91-4adf-a503-3d8ced406143 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:12.350145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231fe77-7e91-4adf-a503-3d8ced406143 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:12.351974 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231fe77-7e91-4adf-a503-3d8ced406143 None admin] RBAC: Adding query filter params (type=network) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:12.352510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231fe77-7e91-4adf-a503-3d8ced406143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:12.355193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231fe77-7e91-4adf-a503-3d8ced406143 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:12.357854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:12.369294 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 384/778] 10.222.0.61 () {56 vars in 1112 bytes} [Wed Sep 7 13:01:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:12.383868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa8f59-fb5f-4463-8cd4-8074f204c346 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:12.601581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa8f59-fb5f-4463-8cd4-8074f204c346 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:12.632807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:12.688027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1048cab-374c-403d-8ad7-c8faa9635a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:12.689089 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1048cab-374c-403d-8ad7-c8faa9635a19 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:12.689589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1048cab-374c-403d-8ad7-c8faa9635a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:12.690069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1048cab-374c-403d-8ad7-c8faa9635a19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:12.691546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1048cab-374c-403d-8ad7-c8faa9635a19 None admin] RBAC: Adding query filter params (interface=public, service_id=37913fd259db4654b9ecac0575638624, region_id=RegionOne) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:12.692085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1048cab-374c-403d-8ad7-c8faa9635a19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:12.694332 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1048cab-374c-403d-8ad7-c8faa9635a19 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:12.702187 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 395/779] 10.222.0.61 () {56 vars in 1251 bytes} [Wed Sep 7 13:01:12 2022] GET /identity/v3/endpoints?service_id=37913fd259db4654b9ecac0575638624&interface=public®ion_id=RegionOne => generated 194 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:12.711137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231fe77-7e91-4adf-a503-3d8ced406143 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:12.943271 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231fe77-7e91-4adf-a503-3d8ced406143 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:13.015875 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa8a7c-ab1e-4cda-bd5b-eaa5716d3094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:13.016900 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa8a7c-ab1e-4cda-bd5b-eaa5716d3094 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:13.017388 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa8a7c-ab1e-4cda-bd5b-eaa5716d3094 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:13.017880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa8a7c-ab1e-4cda-bd5b-eaa5716d3094 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:13.019038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa8a7c-ab1e-4cda-bd5b-eaa5716d3094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:13.021806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa8a7c-ab1e-4cda-bd5b-eaa5716d3094 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:13.031132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 385/780] 10.222.0.61 () {56 vars in 1087 bytes} [Wed Sep 7 13:01:12 2022] GET /identity/v3/services => generated 772 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:14.270841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0822b26-0718-4109-b5ea-ffa2ad684ec5 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:14.271695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0822b26-0718-4109-b5ea-ffa2ad684ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:14.271834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0822b26-0718-4109-b5ea-ffa2ad684ec5 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:14.275555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 396/781] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:14.285550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bc969-4b1a-4c8f-9769-7f0730bdf143 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:14.286310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bc969-4b1a-4c8f-9769-7f0730bdf143 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:14.286480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bc969-4b1a-4c8f-9769-7f0730bdf143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:14.318736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-c80bc969-4b1a-4c8f-9769-7f0730bdf143 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:14.321461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c80bc969-4b1a-4c8f-9769-7f0730bdf143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:14.447798 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:14.467830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 386/782] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:01:14.968334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0822b26-0718-4109-b5ea-ffa2ad684ec5 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:15.078020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48cc68-afc6-46c6-b7ab-5546ddcc3742 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:15.078520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48cc68-afc6-46c6-b7ab-5546ddcc3742 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:15.079014 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48cc68-afc6-46c6-b7ab-5546ddcc3742 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:15.086508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 387/783] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:15.093043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23924e2e-8e2a-48dd-941d-263b34fd9b44 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:15.094075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23924e2e-8e2a-48dd-941d-263b34fd9b44 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:15.094881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23924e2e-8e2a-48dd-941d-263b34fd9b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:15.159866 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-23924e2e-8e2a-48dd-941d-263b34fd9b44 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:15.161140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-23924e2e-8e2a-48dd-941d-263b34fd9b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:15.166456 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0822b26-0718-4109-b5ea-ffa2ad684ec5 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:15.213451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e64277-438b-4f9c-8036-4acb6eebc276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:15.214123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e64277-438b-4f9c-8036-4acb6eebc276 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:15.214385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e64277-438b-4f9c-8036-4acb6eebc276 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:15.214644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e64277-438b-4f9c-8036-4acb6eebc276 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:15.251249 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e64277-438b-4f9c-8036-4acb6eebc276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-e8e64277-438b-4f9c-8036-4acb6eebc276 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:01:15.252311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:15.254232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:15.257271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:15.257271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:15.257271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:15.257271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:15.257271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:01:15.257271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:01:15.257271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:01:15.257271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:01:15.257271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:01:15.257271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:01:15.257271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:01:15.257271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:15.257271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 397/784] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:01:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:15.268005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e64277-438b-4f9c-8036-4acb6eebc276 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:15.369628 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 388/785] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:15.375163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:15.523821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e64277-438b-4f9c-8036-4acb6eebc276 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:15.558840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c07ed-8d75-4a0b-9c8f-423df96ecd09 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:15.559292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c07ed-8d75-4a0b-9c8f-423df96ecd09 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:15.559543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c07ed-8d75-4a0b-9c8f-423df96ecd09 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:15.561529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 389/786] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:15.568860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d4daf-f47e-4894-a845-e38adeb988d9 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:15.569849 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d4daf-f47e-4894-a845-e38adeb988d9 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:15.570315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d4daf-f47e-4894-a845-e38adeb988d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:15.591780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2ba78-91d1-49f5-9eb7-6a88a05e90c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:15.597770 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ba78-91d1-49f5-9eb7-6a88a05e90c5 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:15.598336 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ba78-91d1-49f5-9eb7-6a88a05e90c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:15.598789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ba78-91d1-49f5-9eb7-6a88a05e90c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:15.599767 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2ba78-91d1-49f5-9eb7-6a88a05e90c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:15.600377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2ba78-91d1-49f5-9eb7-6a88a05e90c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:15.601997 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2ba78-91d1-49f5-9eb7-6a88a05e90c5 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:15.609916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-731d4daf-f47e-4894-a845-e38adeb988d9 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:15.611444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-731d4daf-f47e-4894-a845-e38adeb988d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:15.618827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 398/787] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:01:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:15.628258 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2ba78-91d1-49f5-9eb7-6a88a05e90c5 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:15.692957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 390/788] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:15.834969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:15.843461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2ba78-91d1-49f5-9eb7-6a88a05e90c5 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:15.903990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968d04f-acb5-4e19-9d05-50c5bc2fe777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:15.904774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968d04f-acb5-4e19-9d05-50c5bc2fe777 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:15.905047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968d04f-acb5-4e19-9d05-50c5bc2fe777 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:15.905325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968d04f-acb5-4e19-9d05-50c5bc2fe777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:15.921458 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968d04f-acb5-4e19-9d05-50c5bc2fe777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:15.922931 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968d04f-acb5-4e19-9d05-50c5bc2fe777 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-9968d04f-acb5-4e19-9d05-50c5bc2fe777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:15.944799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:15.945803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:15.946709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:15.946709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:15.946709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:15.946709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 399/789] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:15.955944 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d4daf-f47e-4894-a845-e38adeb988d9 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:15.978631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968d04f-acb5-4e19-9d05-50c5bc2fe777 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:16.092184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:16.160280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d4daf-f47e-4894-a845-e38adeb988d9 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:16.182314 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968d04f-acb5-4e19-9d05-50c5bc2fe777 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:16.206734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f22ad-a104-433c-92e8-52fd2ba07efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:16.207710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f22ad-a104-433c-92e8-52fd2ba07efe None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:16.208198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f22ad-a104-433c-92e8-52fd2ba07efe None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:16.208653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f22ad-a104-433c-92e8-52fd2ba07efe None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:16.209832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f22ad-a104-433c-92e8-52fd2ba07efe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:16.210387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f22ad-a104-433c-92e8-52fd2ba07efe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:16.212596 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f22ad-a104-433c-92e8-52fd2ba07efe None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:16.227747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 391/790] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:16.228984 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f22ad-a104-433c-92e8-52fd2ba07efe None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:16.260726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e0ba2-a026-4cdd-942d-827051a1e728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:16.262090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e0ba2-a026-4cdd-942d-827051a1e728 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:16.262672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e0ba2-a026-4cdd-942d-827051a1e728 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:16.263574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e0ba2-a026-4cdd-942d-827051a1e728 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:16.287549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e0ba2-a026-4cdd-942d-827051a1e728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-271e0ba2-a026-4cdd-942d-827051a1e728 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:01:16.298998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:16.300078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:16.300988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:16.300988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:16.300988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:16.300988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:16.300988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:01:16.300988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:01:16.300988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:01:16.300988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:01:16.300988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:01:16.300988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:01:16.300988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:01:16.300988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:16.309821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 400/791] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:01:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:16.311906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e0ba2-a026-4cdd-942d-827051a1e728 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:16.336768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:16.369228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f22ad-a104-433c-92e8-52fd2ba07efe None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:16.388819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:16.429133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93f1f9-9e74-4efb-b22a-1955c54da1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:16.430626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93f1f9-9e74-4efb-b22a-1955c54da1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:16.431275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93f1f9-9e74-4efb-b22a-1955c54da1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:16.431738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93f1f9-9e74-4efb-b22a-1955c54da1f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:16.451028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93f1f9-9e74-4efb-b22a-1955c54da1f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-4a93f1f9-9e74-4efb-b22a-1955c54da1f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:01:16.458642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:16.459828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:16.460663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:16.460663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:16.460663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:16.460663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:16.460663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:01:16.460663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:01:16.460663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:01:16.460663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:01:16.460663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:01:16.460663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:01:16.460663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:01:16.460663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:16.463478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 392/792] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:01:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:16.464670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93f1f9-9e74-4efb-b22a-1955c54da1f7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:16.532951 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e0ba2-a026-4cdd-942d-827051a1e728 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:16.576595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287b3e6-8a76-4bdd-87b1-d5e697b0befe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:16.577247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287b3e6-8a76-4bdd-87b1-d5e697b0befe None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:16.577528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287b3e6-8a76-4bdd-87b1-d5e697b0befe None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:16.577799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287b3e6-8a76-4bdd-87b1-d5e697b0befe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:16.586936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287b3e6-8a76-4bdd-87b1-d5e697b0befe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:16.588444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287b3e6-8a76-4bdd-87b1-d5e697b0befe None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-1287b3e6-8a76-4bdd-87b1-d5e697b0befe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:01:16.600941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:16.601967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:16.602907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:16.602907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:16.602907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:01:16.602907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:01:16.602907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:01:16.602907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:01:16.602907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:01:16.602907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:01:16.602907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:01:16.602907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:16.602907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 401/793] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:01:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:16.604148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287b3e6-8a76-4bdd-87b1-d5e697b0befe None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:16.639645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93f1f9-9e74-4efb-b22a-1955c54da1f7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:16.681506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d90b8-f58e-4b2e-aa90-70807c4af0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:16.682132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d90b8-f58e-4b2e-aa90-70807c4af0df None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:16.682408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d90b8-f58e-4b2e-aa90-70807c4af0df None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:16.682677 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d90b8-f58e-4b2e-aa90-70807c4af0df None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:16.683823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d90b8-f58e-4b2e-aa90-70807c4af0df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:16.683823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d90b8-f58e-4b2e-aa90-70807c4af0df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:16.685098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d90b8-f58e-4b2e-aa90-70807c4af0df None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:16.695504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 393/794] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:01:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:16.696825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f03d4a-d1d2-4f47-9dde-26dcb433c1f9 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:16.697096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f03d4a-d1d2-4f47-9dde-26dcb433c1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:16.697373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f03d4a-d1d2-4f47-9dde-26dcb433c1f9 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:16.703016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 394/795] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:16.703016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f03d4a-d1d2-4f47-9dde-26dcb433c1f9 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:16.733155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:16.751887 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:16.793068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287b3e6-8a76-4bdd-87b1-d5e697b0befe None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:16.831244 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f03d4a-d1d2-4f47-9dde-26dcb433c1f9 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:16.854260 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f27ec-178f-4f88-a322-47185f094c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:16.855721 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f27ec-178f-4f88-a322-47185f094c3e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:16.856148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f27ec-178f-4f88-a322-47185f094c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:16.856559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f27ec-178f-4f88-a322-47185f094c3e None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:16.857668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f27ec-178f-4f88-a322-47185f094c3e None admin] RBAC: Adding query filter params (name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:16.858179 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f27ec-178f-4f88-a322-47185f094c3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:16.860579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f27ec-178f-4f88-a322-47185f094c3e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:16.871387 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 402/796] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:01:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:16.873009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db693ae-aa4e-499e-a6ff-4b4e37f887a3 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:16.873461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db693ae-aa4e-499e-a6ff-4b4e37f887a3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:16.873884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db693ae-aa4e-499e-a6ff-4b4e37f887a3 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:16.876400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 403/797] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:16.877483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db693ae-aa4e-499e-a6ff-4b4e37f887a3 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:16.885401 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856673b-6c7c-48ce-87cf-e708aaad3290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:16.886032 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856673b-6c7c-48ce-87cf-e708aaad3290 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:16.886304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856673b-6c7c-48ce-87cf-e708aaad3290 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:16.886569 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856673b-6c7c-48ce-87cf-e708aaad3290 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:16.887358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856673b-6c7c-48ce-87cf-e708aaad3290 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:16.887663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856673b-6c7c-48ce-87cf-e708aaad3290 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:16.889142 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856673b-6c7c-48ce-87cf-e708aaad3290 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:16.911085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 395/798] 10.222.0.61 () {56 vars in 1135 bytes} [Wed Sep 7 13:01:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:16.912348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccce600-8aa6-4d96-bcce-948a09f259a9 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:16.912629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccce600-8aa6-4d96-bcce-948a09f259a9 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:16.913534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccce600-8aa6-4d96-bcce-948a09f259a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:16.940058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-dccce600-8aa6-4d96-bcce-948a09f259a9 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:16.941675 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-dccce600-8aa6-4d96-bcce-948a09f259a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:17.026935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db693ae-aa4e-499e-a6ff-4b4e37f887a3 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:17.038396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 396/799] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:17.039317 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccce600-8aa6-4d96-bcce-948a09f259a9 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:17.084653 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:17.094450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb887185-f547-4978-a297-017584e235fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:17.095137 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb887185-f547-4978-a297-017584e235fc None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:17.095462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb887185-f547-4978-a297-017584e235fc None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:17.095767 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb887185-f547-4978-a297-017584e235fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:17.096332 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:17.110743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb887185-f547-4978-a297-017584e235fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:17.112698 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb887185-f547-4978-a297-017584e235fc None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-bb887185-f547-4978-a297-017584e235fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:17.131059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:17.132202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:17.133132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:17.133132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:17.133132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:17.134772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 404/800] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:17.136310 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca9fdb-ba7e-4b11-99e7-32dd339e650f None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:17.136758 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca9fdb-ba7e-4b11-99e7-32dd339e650f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:17.137201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca9fdb-ba7e-4b11-99e7-32dd339e650f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:17.164262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-8aca9fdb-ba7e-4b11-99e7-32dd339e650f None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:17.166611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-8aca9fdb-ba7e-4b11-99e7-32dd339e650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:17.262790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccce600-8aa6-4d96-bcce-948a09f259a9 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:17.280507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 405/801] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:17.280507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca9fdb-ba7e-4b11-99e7-32dd339e650f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:17.330334 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8b1a8-0f18-4851-b772-b2acfa33bb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:17.331124 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8b1a8-0f18-4851-b772-b2acfa33bb8a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:17.331610 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8b1a8-0f18-4851-b772-b2acfa33bb8a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:17.332066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8b1a8-0f18-4851-b772-b2acfa33bb8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:17.348164 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8b1a8-0f18-4851-b772-b2acfa33bb8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:17.350868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8b1a8-0f18-4851-b772-b2acfa33bb8a None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-ffb8b1a8-0f18-4851-b772-b2acfa33bb8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:17.377560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:17.379479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:17.381299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:17.381299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:17.381299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:17.381299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 397/802] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:17.381299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8b1a8-0f18-4851-b772-b2acfa33bb8a None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:17.412201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca9fdb-ba7e-4b11-99e7-32dd339e650f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:17.430917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:17.458764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea132f-d6e1-4ce4-acb8-59613ea6652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:17.459462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea132f-d6e1-4ce4-acb8-59613ea6652e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:17.460357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea132f-d6e1-4ce4-acb8-59613ea6652e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:17.460648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea132f-d6e1-4ce4-acb8-59613ea6652e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:17.473573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea132f-d6e1-4ce4-acb8-59613ea6652e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:17.475208 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea132f-d6e1-4ce4-acb8-59613ea6652e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-62ea132f-d6e1-4ce4-acb8-59613ea6652e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:17.499520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:17.501817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:17.504058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:17.504058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:17.504058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:17.509691 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 406/803] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:17.509691 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea132f-d6e1-4ce4-acb8-59613ea6652e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:17.527941 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:17.583296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8b1a8-0f18-4851-b772-b2acfa33bb8a None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:17.632537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b2eb8-8c6f-438e-8030-dde8ad3f5238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:17.633643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b2eb8-8c6f-438e-8030-dde8ad3f5238 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:17.634055 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b2eb8-8c6f-438e-8030-dde8ad3f5238 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:17.634462 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b2eb8-8c6f-438e-8030-dde8ad3f5238 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:17.635298 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b2eb8-8c6f-438e-8030-dde8ad3f5238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:17.635693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b2eb8-8c6f-438e-8030-dde8ad3f5238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:17.637869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b2eb8-8c6f-438e-8030-dde8ad3f5238 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:17.653482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea132f-d6e1-4ce4-acb8-59613ea6652e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:17.654791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 398/804] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:17.655864 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b2eb8-8c6f-438e-8030-dde8ad3f5238 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:17.712390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b600b5d-a623-4bac-807b-5d9182aab0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:17.713306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b600b5d-a623-4bac-807b-5d9182aab0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:17.713746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b600b5d-a623-4bac-807b-5d9182aab0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:17.714183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b600b5d-a623-4bac-807b-5d9182aab0c6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:17.715182 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b600b5d-a623-4bac-807b-5d9182aab0c6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:17.717567 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b600b5d-a623-4bac-807b-5d9182aab0c6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-7b600b5d-a623-4bac-807b-5d9182aab0c6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:17.733148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:17.734730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:01:17.735647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:17.736845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 407/805] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:17 2022] GET /identity/v3/services/network => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:17.738132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b600b5d-a623-4bac-807b-5d9182aab0c6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:17.782199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:17.818037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b2eb8-8c6f-438e-8030-dde8ad3f5238 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:17.874018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f820462-3a2d-425b-89cd-9ca7d70b0f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:17.874902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f820462-3a2d-425b-89cd-9ca7d70b0f79 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:17.875647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f820462-3a2d-425b-89cd-9ca7d70b0f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:17.875981 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f820462-3a2d-425b-89cd-9ca7d70b0f79 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:17.876789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f820462-3a2d-425b-89cd-9ca7d70b0f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:17.877149 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f820462-3a2d-425b-89cd-9ca7d70b0f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:17.880843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f820462-3a2d-425b-89cd-9ca7d70b0f79 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:17.883664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:17.891201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b600b5d-a623-4bac-807b-5d9182aab0c6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:17.901171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 399/806] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:17.902268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f820462-3a2d-425b-89cd-9ca7d70b0f79 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:17.940188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefa7f7-f612-43d7-b3d4-14668e18c0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:17.941058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefa7f7-f612-43d7-b3d4-14668e18c0cb None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:17.941434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefa7f7-f612-43d7-b3d4-14668e18c0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:17.941640 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefa7f7-f612-43d7-b3d4-14668e18c0cb None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:17.942366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefa7f7-f612-43d7-b3d4-14668e18c0cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:17.942743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefa7f7-f612-43d7-b3d4-14668e18c0cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:17.944887 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefa7f7-f612-43d7-b3d4-14668e18c0cb None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:17.959534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 408/807] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:17.961059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefa7f7-f612-43d7-b3d4-14668e18c0cb None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:18.062436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f820462-3a2d-425b-89cd-9ca7d70b0f79 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:18.063937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:18.078307 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefa7f7-f612-43d7-b3d4-14668e18c0cb None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:18.111270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d37d2f-8ce5-4029-a317-a7a0f14e8ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:18.111919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d37d2f-8ce5-4029-a317-a7a0f14e8ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:18.112196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d37d2f-8ce5-4029-a317-a7a0f14e8ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:18.112492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d37d2f-8ce5-4029-a317-a7a0f14e8ae2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:18.113094 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d37d2f-8ce5-4029-a317-a7a0f14e8ae2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:18.114505 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d37d2f-8ce5-4029-a317-a7a0f14e8ae2 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-b0d37d2f-8ce5-4029-a317-a7a0f14e8ae2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:18.124592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:18.127097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:18.128857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 400/808] 10.222.0.61 () {56 vars in 1129 bytes} [Wed Sep 7 13:01:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:18.130406 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d37d2f-8ce5-4029-a317-a7a0f14e8ae2 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:18.137438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ae4af-27d0-4258-ace4-a1ebf462bae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:18.138398 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ae4af-27d0-4258-ace4-a1ebf462bae5 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:18.138844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ae4af-27d0-4258-ace4-a1ebf462bae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:18.139334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ae4af-27d0-4258-ace4-a1ebf462bae5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:18.147603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ae4af-27d0-4258-ace4-a1ebf462bae5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:18.149150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ae4af-27d0-4258-ace4-a1ebf462bae5 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-6a9ae4af-27d0-4258-ace4-a1ebf462bae5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:01:18.161872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:18.164753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:18.169147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:18.169147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:18.169147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:01:18.169147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:01:18.169147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:01:18.169147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:01:18.169147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:01:18.169147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:01:18.169147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:01:18.169147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:18.169147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 409/809] 10.222.0.61 () {56 vars in 1093 bytes} [Wed Sep 7 13:01:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:18.169147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ae4af-27d0-4258-ace4-a1ebf462bae5 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:18.227245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:18.275763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d37d2f-8ce5-4029-a317-a7a0f14e8ae2 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:18.297478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ae4af-27d0-4258-ace4-a1ebf462bae5 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:18.317892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d1ca2-f63a-4242-a709-3a1d342c5527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:18.318959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d1ca2-f63a-4242-a709-3a1d342c5527 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:18.319956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d1ca2-f63a-4242-a709-3a1d342c5527 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:18.320425 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d1ca2-f63a-4242-a709-3a1d342c5527 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:18.321496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d1ca2-f63a-4242-a709-3a1d342c5527 None admin] RBAC: Adding query filter params (name=network) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:18.321987 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d1ca2-f63a-4242-a709-3a1d342c5527 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:18.324432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d1ca2-f63a-4242-a709-3a1d342c5527 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:18.332420 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 401/810] 10.222.0.61 () {56 vars in 1112 bytes} [Wed Sep 7 13:01:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:18.333747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d1ca2-f63a-4242-a709-3a1d342c5527 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:18.358314 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a20e2-91a2-4492-a32e-c4c779c303a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:18.358960 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a20e2-91a2-4492-a32e-c4c779c303a1 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:18.359283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a20e2-91a2-4492-a32e-c4c779c303a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:18.359563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a20e2-91a2-4492-a32e-c4c779c303a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:18.369648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a20e2-91a2-4492-a32e-c4c779c303a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:18.371182 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a20e2-91a2-4492-a32e-c4c779c303a1 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-091a20e2-91a2-4492-a32e-c4c779c303a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:01:18.386017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:18.387775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:18.389379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:18.389379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:18.389379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:01:18.389379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:01:18.389379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:01:18.389379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:01:18.389379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:01:18.389379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:01:18.389379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:01:18.389379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:18.389379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 410/811] 10.222.0.61 () {56 vars in 1117 bytes} [Wed Sep 7 13:01:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:18.389379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a20e2-91a2-4492-a32e-c4c779c303a1 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:18.402897 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:18.487418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d1ca2-f63a-4242-a709-3a1d342c5527 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:18.513502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:18.528519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33b1f0-2d6d-4535-9727-324590e814b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:18.529486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33b1f0-2d6d-4535-9727-324590e814b0 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:18.529486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33b1f0-2d6d-4535-9727-324590e814b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:18.529779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33b1f0-2d6d-4535-9727-324590e814b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:18.529912 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a20e2-91a2-4492-a32e-c4c779c303a1 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:18.537832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33b1f0-2d6d-4535-9727-324590e814b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:18.539460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33b1f0-2d6d-4535-9727-324590e814b0 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-1f33b1f0-2d6d-4535-9727-324590e814b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:01:18.560362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:18.562281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:18.570544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:01:18.570544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:01:18.570544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:01:18.570544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:01:18.570544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:01:18.570544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:01:18.570544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:18.570544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:18.570544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 402/812] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:18.570544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33b1f0-2d6d-4535-9727-324590e814b0 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:18.571744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d37e20-e59f-45ae-a43d-134be4aae1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:18.571744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d37e20-e59f-45ae-a43d-134be4aae1bd None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:18.571744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d37e20-e59f-45ae-a43d-134be4aae1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:18.571744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d37e20-e59f-45ae-a43d-134be4aae1bd None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:18.572303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d37e20-e59f-45ae-a43d-134be4aae1bd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:18.572303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d37e20-e59f-45ae-a43d-134be4aae1bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:18.573162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d37e20-e59f-45ae-a43d-134be4aae1bd None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:18.578318 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 411/813] 10.222.0.61 () {56 vars in 1124 bytes} [Wed Sep 7 13:01:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:18.579222 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d37e20-e59f-45ae-a43d-134be4aae1bd None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:18.669820 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:18.701918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d37e20-e59f-45ae-a43d-134be4aae1bd None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:18.737664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33b1f0-2d6d-4535-9727-324590e814b0 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:18.750978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e071bb5-2dd1-42c8-9a28-8b3992263763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:18.751918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e071bb5-2dd1-42c8-9a28-8b3992263763 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:18.752192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e071bb5-2dd1-42c8-9a28-8b3992263763 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:18.752466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e071bb5-2dd1-42c8-9a28-8b3992263763 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:18.753127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e071bb5-2dd1-42c8-9a28-8b3992263763 None admin] RBAC: Adding query filter params (type=network) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:18.753436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e071bb5-2dd1-42c8-9a28-8b3992263763 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:18.754851 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e071bb5-2dd1-42c8-9a28-8b3992263763 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:18.764483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 412/814] 10.222.0.61 () {56 vars in 1112 bytes} [Wed Sep 7 13:01:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:18.766111 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e071bb5-2dd1-42c8-9a28-8b3992263763 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:18.787075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd59a9-00bb-4f08-8c37-3e7b976e5ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:18.788230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd59a9-00bb-4f08-8c37-3e7b976e5ade None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:18.788506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd59a9-00bb-4f08-8c37-3e7b976e5ade None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:18.788775 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd59a9-00bb-4f08-8c37-3e7b976e5ade None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:18.789529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd59a9-00bb-4f08-8c37-3e7b976e5ade None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:18.789840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd59a9-00bb-4f08-8c37-3e7b976e5ade None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:18.791347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd59a9-00bb-4f08-8c37-3e7b976e5ade None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:18.811756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 403/815] 10.222.0.61 () {56 vars in 1133 bytes} [Wed Sep 7 13:01:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:18.813221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd59a9-00bb-4f08-8c37-3e7b976e5ade None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:18.878939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:18.920745 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e071bb5-2dd1-42c8-9a28-8b3992263763 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:18.951656 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd59a9-00bb-4f08-8c37-3e7b976e5ade None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:18.964755 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afd3fe-4c54-4093-8645-fed6916d0fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:18.965524 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afd3fe-4c54-4093-8645-fed6916d0fea None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:18.965995 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afd3fe-4c54-4093-8645-fed6916d0fea None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:18.966324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afd3fe-4c54-4093-8645-fed6916d0fea None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:18.967356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afd3fe-4c54-4093-8645-fed6916d0fea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:18.967732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afd3fe-4c54-4093-8645-fed6916d0fea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:18.969873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afd3fe-4c54-4093-8645-fed6916d0fea None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:18.981065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:18.994996 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28363a62-62bf-4acc-a54a-a116ff286900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:18.995659 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28363a62-62bf-4acc-a54a-a116ff286900 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:18.995936 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28363a62-62bf-4acc-a54a-a116ff286900 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:19.003258 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 413/816] 10.222.0.61 () {56 vars in 1149 bytes} [Wed Sep 7 13:01:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:19.003632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28363a62-62bf-4acc-a54a-a116ff286900 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:19.003632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28363a62-62bf-4acc-a54a-a116ff286900 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:19.003906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afd3fe-4c54-4093-8645-fed6916d0fea None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:19.004028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28363a62-62bf-4acc-a54a-a116ff286900 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:19.004028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28363a62-62bf-4acc-a54a-a116ff286900 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:19.016988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 404/817] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:01:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:19.017953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28363a62-62bf-4acc-a54a-a116ff286900 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:19.142641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afd3fe-4c54-4093-8645-fed6916d0fea None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:19.151971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:19.155473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28363a62-62bf-4acc-a54a-a116ff286900 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:19.198694 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf52ef-a2a3-432c-a209-a38937e711d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:19.199358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf52ef-a2a3-432c-a209-a38937e711d7 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:19.199637 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf52ef-a2a3-432c-a209-a38937e711d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:19.199911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf52ef-a2a3-432c-a209-a38937e711d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:19.200608 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf52ef-a2a3-432c-a209-a38937e711d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:19.202050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf52ef-a2a3-432c-a209-a38937e711d7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:19.209787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699aec42-e021-4ed7-9477-55494d50166e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:19.210862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699aec42-e021-4ed7-9477-55494d50166e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:19.211390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699aec42-e021-4ed7-9477-55494d50166e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:19.211887 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699aec42-e021-4ed7-9477-55494d50166e None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:19.212974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699aec42-e021-4ed7-9477-55494d50166e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:19.213498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699aec42-e021-4ed7-9477-55494d50166e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:19.216137 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699aec42-e021-4ed7-9477-55494d50166e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:19.227150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 414/818] 10.222.0.61 () {56 vars in 1124 bytes} [Wed Sep 7 13:01:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:19.228658 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699aec42-e021-4ed7-9477-55494d50166e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:19.229041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 405/819] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:01:19 2022] POST /identity/v3/endpoints => generated 337 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:01:19.230834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf52ef-a2a3-432c-a209-a38937e711d7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:19.354437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:19.425247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699aec42-e021-4ed7-9477-55494d50166e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:19.433986 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf52ef-a2a3-432c-a209-a38937e711d7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:19.487787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b5e36-05b3-46e7-a129-6a0c36e0ba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:19.488696 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b5e36-05b3-46e7-a129-6a0c36e0ba55 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:19.489065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b5e36-05b3-46e7-a129-6a0c36e0ba55 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:19.489508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b5e36-05b3-46e7-a129-6a0c36e0ba55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:19.492676 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec423a8-a644-4e84-a077-700028091686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:19.493343 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec423a8-a644-4e84-a077-700028091686 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:19.493623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec423a8-a644-4e84-a077-700028091686 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:19.493897 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec423a8-a644-4e84-a077-700028091686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:19.513606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec423a8-a644-4e84-a077-700028091686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:19.516237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec423a8-a644-4e84-a077-700028091686 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-fec423a8-a644-4e84-a077-700028091686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:19.541179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:19.543736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:19.546209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:19.546209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:19.546209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:19.546209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 406/820] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:19.546209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec423a8-a644-4e84-a077-700028091686 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:19.550857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:19.622146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b5e36-05b3-46e7-a129-6a0c36e0ba55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:19.624882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b5e36-05b3-46e7-a129-6a0c36e0ba55 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:19.673977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec423a8-a644-4e84-a077-700028091686 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:19.714878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:19.725605 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952870b8-ea86-4c05-99bd-407d0747990b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:19.726499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952870b8-ea86-4c05-99bd-407d0747990b None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:19.726889 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952870b8-ea86-4c05-99bd-407d0747990b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:19.727360 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952870b8-ea86-4c05-99bd-407d0747990b None admin] PATH_INFO: `/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/0f886e0999d34bf9870b448772cc38a3/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:19.730185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952870b8-ea86-4c05-99bd-407d0747990b None admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user_id=0f886e0999d34bf9870b448772cc38a3, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:19.732077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952870b8-ea86-4c05-99bd-407d0747990b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:19.753153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 407/821] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:01:19 2022] PUT /identity/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/0f886e0999d34bf9870b448772cc38a3/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:01:19.754941 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952870b8-ea86-4c05-99bd-407d0747990b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:19.775245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 415/822] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:01:19 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:01:19.776809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b5e36-05b3-46e7-a129-6a0c36e0ba55 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:19.897211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:19.974577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b5e36-05b3-46e7-a129-6a0c36e0ba55 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:19.975384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952870b8-ea86-4c05-99bd-407d0747990b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:20.026729 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f987594-1eea-432f-9330-1fdb72749787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:20.027396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f987594-1eea-432f-9330-1fdb72749787 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:20.027693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f987594-1eea-432f-9330-1fdb72749787 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:20.028087 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f987594-1eea-432f-9330-1fdb72749787 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:20.028779 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f987594-1eea-432f-9330-1fdb72749787 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:20.029066 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f987594-1eea-432f-9330-1fdb72749787 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:20.030482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f987594-1eea-432f-9330-1fdb72749787 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:20.042734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80075f47-b293-40ae-9fc5-2e463e7ebafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:20.043833 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80075f47-b293-40ae-9fc5-2e463e7ebafe None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:20.044438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80075f47-b293-40ae-9fc5-2e463e7ebafe None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:20.044825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80075f47-b293-40ae-9fc5-2e463e7ebafe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:20.051389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 416/823] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:20.051679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80075f47-b293-40ae-9fc5-2e463e7ebafe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:20.051857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f987594-1eea-432f-9330-1fdb72749787 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:20.052009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80075f47-b293-40ae-9fc5-2e463e7ebafe None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:20.082210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:20.095806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 408/824] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:01:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:01:20.097577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80075f47-b293-40ae-9fc5-2e463e7ebafe None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:20.254976 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f987594-1eea-432f-9330-1fdb72749787 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:20.340757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485dccf-dca5-41ca-876f-a615360ff9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:20.341923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485dccf-dca5-41ca-876f-a615360ff9ba None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:20.342377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485dccf-dca5-41ca-876f-a615360ff9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:20.342817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485dccf-dca5-41ca-876f-a615360ff9ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:20.344108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80075f47-b293-40ae-9fc5-2e463e7ebafe None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:20.367295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485dccf-dca5-41ca-876f-a615360ff9ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:20.370492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485dccf-dca5-41ca-876f-a615360ff9ba None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:20.387077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-a485dccf-dca5-41ca-876f-a615360ff9ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:20.396386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:20.397457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:20.398365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:20.398365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:20.398365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:20.400420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 417/825] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:20.407016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485dccf-dca5-41ca-876f-a615360ff9ba None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:20.419309 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb88fa0-28e7-4d72-aecd-da201819c0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:20.420456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb88fa0-28e7-4d72-aecd-da201819c0eb None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:20.420953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb88fa0-28e7-4d72-aecd-da201819c0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:20.421452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb88fa0-28e7-4d72-aecd-da201819c0eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:20.435695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb88fa0-28e7-4d72-aecd-da201819c0eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:20.438548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb88fa0-28e7-4d72-aecd-da201819c0eb None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-efb88fa0-28e7-4d72-aecd-da201819c0eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:01:20.463597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:20.464542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:20.465430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:01:20.465430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:01:20.465430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:01:20.465430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:01:20.465430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:01:20.465430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:01:20.465430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:20.465430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:20.466566 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 409/826] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:20.476548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb88fa0-28e7-4d72-aecd-da201819c0eb None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:20.605813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:20.613581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485dccf-dca5-41ca-876f-a615360ff9ba None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:20.699901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8dbe94-86a7-44a1-a960-7fccf1882784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:20.701918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8dbe94-86a7-44a1-a960-7fccf1882784 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:20.702618 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8dbe94-86a7-44a1-a960-7fccf1882784 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:20.703331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8dbe94-86a7-44a1-a960-7fccf1882784 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:20.704611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8dbe94-86a7-44a1-a960-7fccf1882784 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:20.705175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8dbe94-86a7-44a1-a960-7fccf1882784 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:20.706262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb88fa0-28e7-4d72-aecd-da201819c0eb None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:20.707923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8dbe94-86a7-44a1-a960-7fccf1882784 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:20.738153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 418/827] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:20.743947 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8dbe94-86a7-44a1-a960-7fccf1882784 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:20.772746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a687c8-d342-499b-85ef-ee46428ccda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:20.773384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a687c8-d342-499b-85ef-ee46428ccda5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:20.773670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a687c8-d342-499b-85ef-ee46428ccda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:20.773944 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a687c8-d342-499b-85ef-ee46428ccda5 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:20.774674 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a687c8-d342-499b-85ef-ee46428ccda5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:20.774980 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a687c8-d342-499b-85ef-ee46428ccda5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:20.776713 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a687c8-d342-499b-85ef-ee46428ccda5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:20.793066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 410/828] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:01:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:20.802718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a687c8-d342-499b-85ef-ee46428ccda5 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:20.886648 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:20.947888 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8dbe94-86a7-44a1-a960-7fccf1882784 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:20.969858 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a687c8-d342-499b-85ef-ee46428ccda5 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:21.008334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:21.020955 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaa399-4a38-458d-a093-cbd5b9ee0fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:21.021644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaa399-4a38-458d-a093-cbd5b9ee0fea None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:21.021968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaa399-4a38-458d-a093-cbd5b9ee0fea None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:21.022270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaa399-4a38-458d-a093-cbd5b9ee0fea None admin] PATH_INFO: `/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/d71863f18330414f9b6238b7e8903701/roles/5f8e244e8257481e85e65a504aa4bb74` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:21.025810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900a726-997a-48b9-a198-0a33f68244b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:21.027238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900a726-997a-48b9-a198-0a33f68244b7 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:21.027948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900a726-997a-48b9-a198-0a33f68244b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:21.028840 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900a726-997a-48b9-a198-0a33f68244b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:21.040697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaa399-4a38-458d-a093-cbd5b9ee0fea None admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user_id=d71863f18330414f9b6238b7e8903701, role_id=5f8e244e8257481e85e65a504aa4bb74)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:21.042504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaa399-4a38-458d-a093-cbd5b9ee0fea None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:21.043587 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900a726-997a-48b9-a198-0a33f68244b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:21.046546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900a726-997a-48b9-a198-0a33f68244b7 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:21.067162 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 411/829] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:01:20 2022] PUT /identity/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/d71863f18330414f9b6238b7e8903701/roles/5f8e244e8257481e85e65a504aa4bb74 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-a900a726-997a-48b9-a198-0a33f68244b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:01:21.072140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:21.074139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:21.081788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:01:21.081788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:01:21.081788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:01:21.081788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:01:21.081788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:01:21.081788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:01:21.081788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:21.081788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:21.081788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 419/830] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:21.085307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaa399-4a38-458d-a093-cbd5b9ee0fea None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:21.208925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:21.223091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaa399-4a38-458d-a093-cbd5b9ee0fea None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:21.267398 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47572cd-e2d3-47e3-bd4c-1a6c08a73316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:21.268185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47572cd-e2d3-47e3-bd4c-1a6c08a73316 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:21.268471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47572cd-e2d3-47e3-bd4c-1a6c08a73316 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:21.268738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47572cd-e2d3-47e3-bd4c-1a6c08a73316 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:21.269458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47572cd-e2d3-47e3-bd4c-1a6c08a73316 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:21.269768 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47572cd-e2d3-47e3-bd4c-1a6c08a73316 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:21.271430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47572cd-e2d3-47e3-bd4c-1a6c08a73316 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:21.286050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 412/831] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:01:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:21.292022 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900a726-997a-48b9-a198-0a33f68244b7 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:21.538168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900a726-997a-48b9-a198-0a33f68244b7 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:21.575964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d8e3f-9862-42e8-8e01-4f80c26e46cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:21.576672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d8e3f-9862-42e8-8e01-4f80c26e46cc None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:21.576960 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d8e3f-9862-42e8-8e01-4f80c26e46cc None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:21.577225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d8e3f-9862-42e8-8e01-4f80c26e46cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:21.577979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d8e3f-9862-42e8-8e01-4f80c26e46cc None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=ba93e2d4aaed4c5595d5fd7363626bcb) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:21.578291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d8e3f-9862-42e8-8e01-4f80c26e46cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:21.579787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d8e3f-9862-42e8-8e01-4f80c26e46cc None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:21.587417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 420/832] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:01:21 2022] GET /identity/v3/role_assignments?user.id=ba93e2d4aaed4c5595d5fd7363626bcb&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:22.305481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4049ba-85eb-4354-bfe3-5f1b98afb237 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:22.306287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4049ba-85eb-4354-bfe3-5f1b98afb237 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:22.306287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4049ba-85eb-4354-bfe3-5f1b98afb237 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:22.307814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 413/833] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:22.314020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d14b4c-19e3-448d-bd38-e1ee5eb63a85 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:22.314754 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d14b4c-19e3-448d-bd38-e1ee5eb63a85 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:22.314754 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d14b4c-19e3-448d-bd38-e1ee5eb63a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:22.334591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-16d14b4c-19e3-448d-bd38-e1ee5eb63a85 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:22.335757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-16d14b4c-19e3-448d-bd38-e1ee5eb63a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:22.393575 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:22.418640 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 421/834] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:01:22.596252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b522e-6cb4-405e-a74d-1f8aed5a5d24 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:22.596490 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b522e-6cb4-405e-a74d-1f8aed5a5d24 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:22.596859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b522e-6cb4-405e-a74d-1f8aed5a5d24 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:22.598466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 414/835] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:22.604450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc9d80-0b5c-46be-bbcf-3796677f9b75 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:22.604715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc9d80-0b5c-46be-bbcf-3796677f9b75 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:22.604998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc9d80-0b5c-46be-bbcf-3796677f9b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:22.625471 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-0fbc9d80-0b5c-46be-bbcf-3796677f9b75 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:22.626798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-0fbc9d80-0b5c-46be-bbcf-3796677f9b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:22.715313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 422/836] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:22.788014 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b522e-6cb4-405e-a74d-1f8aed5a5d24 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:22.917176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b522e-6cb4-405e-a74d-1f8aed5a5d24 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:22.960786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4c8a5-ed92-4a58-974f-dcc25b4e0fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:22.961469 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4c8a5-ed92-4a58-974f-dcc25b4e0fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:22.961749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4c8a5-ed92-4a58-974f-dcc25b4e0fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:22.962096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4c8a5-ed92-4a58-974f-dcc25b4e0fe1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:22.977215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4c8a5-ed92-4a58-974f-dcc25b4e0fe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-4bc4c8a5-ed92-4a58-974f-dcc25b4e0fe1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:01:22.982402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:22.984120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:22.985607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:22.985607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:22.985607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:22.985607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:22.985607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:01:22.985607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:01:22.985607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:01:22.985607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:01:22.985607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:01:22.985607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:01:22.985607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:01:22.985607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:22.985607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 415/837] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:01:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:22.997839 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc9d80-0b5c-46be-bbcf-3796677f9b75 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:22.998263 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4c8a5-ed92-4a58-974f-dcc25b4e0fe1 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:23.015173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:23.123927 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4c8a5-ed92-4a58-974f-dcc25b4e0fe1 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:23.165783 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07635f74-f552-44e6-a388-e2943dfd55cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:23.166914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07635f74-f552-44e6-a388-e2943dfd55cf None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:23.167363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07635f74-f552-44e6-a388-e2943dfd55cf None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:23.167627 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07635f74-f552-44e6-a388-e2943dfd55cf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:23.168266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07635f74-f552-44e6-a388-e2943dfd55cf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:23.169951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07635f74-f552-44e6-a388-e2943dfd55cf None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-07635f74-f552-44e6-a388-e2943dfd55cf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:23.180794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:23.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:23.184510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 416/838] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:01:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:23.191005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07635f74-f552-44e6-a388-e2943dfd55cf None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:23.225060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc9d80-0b5c-46be-bbcf-3796677f9b75 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:23.253748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:23.293439 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b833b18-7b38-45dc-9086-da78a8a8eaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:23.294513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b833b18-7b38-45dc-9086-da78a8a8eaf6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:23.295385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b833b18-7b38-45dc-9086-da78a8a8eaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:23.295504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b833b18-7b38-45dc-9086-da78a8a8eaf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:23.296672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b833b18-7b38-45dc-9086-da78a8a8eaf6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:23.297307 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b833b18-7b38-45dc-9086-da78a8a8eaf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:23.299991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b833b18-7b38-45dc-9086-da78a8a8eaf6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:23.308184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:23.319795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07635f74-f552-44e6-a388-e2943dfd55cf None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:23.323880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 423/839] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:01:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:23.326286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b833b18-7b38-45dc-9086-da78a8a8eaf6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:23.364434 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e29d2-c829-4706-a825-5abf1a06d94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:23.365543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e29d2-c829-4706-a825-5abf1a06d94e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:23.366151 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e29d2-c829-4706-a825-5abf1a06d94e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:23.366639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e29d2-c829-4706-a825-5abf1a06d94e None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:23.367574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e29d2-c829-4706-a825-5abf1a06d94e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:23.368089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e29d2-c829-4706-a825-5abf1a06d94e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:23.369870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e29d2-c829-4706-a825-5abf1a06d94e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:23.376378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 417/840] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:01:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:23.383010 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e29d2-c829-4706-a825-5abf1a06d94e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:23.497736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b833b18-7b38-45dc-9086-da78a8a8eaf6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:23.551379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bc0d3-c76d-447f-94ac-4d9bdcf75643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:23.552098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bc0d3-c76d-447f-94ac-4d9bdcf75643 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:23.552599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bc0d3-c76d-447f-94ac-4d9bdcf75643 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:23.553145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bc0d3-c76d-447f-94ac-4d9bdcf75643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:23.569248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e29d2-c829-4706-a825-5abf1a06d94e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:23.575542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bc0d3-c76d-447f-94ac-4d9bdcf75643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:23.578478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bc0d3-c76d-447f-94ac-4d9bdcf75643 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-710bc0d3-c76d-447f-94ac-4d9bdcf75643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:23.608875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:23.609899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:23.610794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:23.610794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:23.610794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:23.616275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 424/841] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:23.621469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bc0d3-c76d-447f-94ac-4d9bdcf75643 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:23.622937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7125503-831b-438d-b993-f1e181b1014f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:23.623963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7125503-831b-438d-b993-f1e181b1014f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:23.624451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7125503-831b-438d-b993-f1e181b1014f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:23.624921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7125503-831b-438d-b993-f1e181b1014f None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:23.626051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7125503-831b-438d-b993-f1e181b1014f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:23.626564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7125503-831b-438d-b993-f1e181b1014f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:23.628624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7125503-831b-438d-b993-f1e181b1014f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:23.643561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 418/842] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:01:23 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:23.684851 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:23.810384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bc0d3-c76d-447f-94ac-4d9bdcf75643 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:23.854708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585d81a-c1ff-40d9-912a-9388b3eb3691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:23.855773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585d81a-c1ff-40d9-912a-9388b3eb3691 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:23.856296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585d81a-c1ff-40d9-912a-9388b3eb3691 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:23.857002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585d81a-c1ff-40d9-912a-9388b3eb3691 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:23.858099 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585d81a-c1ff-40d9-912a-9388b3eb3691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:23.858653 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585d81a-c1ff-40d9-912a-9388b3eb3691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:23.860415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585d81a-c1ff-40d9-912a-9388b3eb3691 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:23.883431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 425/843] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:23.893107 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7125503-831b-438d-b993-f1e181b1014f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:23.899816 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:24.115430 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8f1c0-771f-4f65-8ebf-f96cf9727306 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:24.116436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8f1c0-771f-4f65-8ebf-f96cf9727306 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:24.116756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8f1c0-771f-4f65-8ebf-f96cf9727306 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:24.116862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7125503-831b-438d-b993-f1e181b1014f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:24.118490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 426/844] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:24.124319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576480ec-9910-4dea-8516-7cf960ce7020 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:24.124620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576480ec-9910-4dea-8516-7cf960ce7020 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:24.124878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576480ec-9910-4dea-8516-7cf960ce7020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:24.145591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-576480ec-9910-4dea-8516-7cf960ce7020 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:24.146855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-576480ec-9910-4dea-8516-7cf960ce7020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:24.182624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b79ba-ac25-4ce3-967f-9af1624212e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:24.183235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b79ba-ac25-4ce3-967f-9af1624212e6 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:24.183523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b79ba-ac25-4ce3-967f-9af1624212e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:24.183801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b79ba-ac25-4ce3-967f-9af1624212e6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:24.193051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b79ba-ac25-4ce3-967f-9af1624212e6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:24.195599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b79ba-ac25-4ce3-967f-9af1624212e6 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-178b79ba-ac25-4ce3-967f-9af1624212e6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:01:24.208792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:24.210449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:24.211611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:24.211611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:24.211611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:01:24.211611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:01:24.211611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:01:24.211611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:01:24.211611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:01:24.211611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:01:24.211611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:01:24.211611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:24.211611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 419/845] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:01:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:24.220902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b79ba-ac25-4ce3-967f-9af1624212e6 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:24.247246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 427/846] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:24.357036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:24.442992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b79ba-ac25-4ce3-967f-9af1624212e6 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:24.508223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecef44-2381-4cae-92cb-dc7c6bace748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:24.509246 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecef44-2381-4cae-92cb-dc7c6bace748 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:24.509728 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecef44-2381-4cae-92cb-dc7c6bace748 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:24.510200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecef44-2381-4cae-92cb-dc7c6bace748 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:24.511691 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecef44-2381-4cae-92cb-dc7c6bace748 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:24.512240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecef44-2381-4cae-92cb-dc7c6bace748 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:24.514983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecef44-2381-4cae-92cb-dc7c6bace748 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:24.553932 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 420/847] 10.222.0.61 () {56 vars in 1135 bytes} [Wed Sep 7 13:01:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:24.563049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576480ec-9910-4dea-8516-7cf960ce7020 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:24.600662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:24.664153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecef44-2381-4cae-92cb-dc7c6bace748 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:24.771979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576480ec-9910-4dea-8516-7cf960ce7020 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:24.841373 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96160dd2-5c64-45e6-8bfd-6b995bce3a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:24.842757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96160dd2-5c64-45e6-8bfd-6b995bce3a39 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:24.843495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96160dd2-5c64-45e6-8bfd-6b995bce3a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:24.844352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96160dd2-5c64-45e6-8bfd-6b995bce3a39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:24.875791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecef44-2381-4cae-92cb-dc7c6bace748 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:24.905221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:24.926414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857bdb2-cb5c-4050-9049-fbc004ce68d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:24.926972 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857bdb2-cb5c-4050-9049-fbc004ce68d4 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:24.927300 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857bdb2-cb5c-4050-9049-fbc004ce68d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:24.927578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857bdb2-cb5c-4050-9049-fbc004ce68d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:24.944377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857bdb2-cb5c-4050-9049-fbc004ce68d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-3857bdb2-cb5c-4050-9049-fbc004ce68d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:01:24.950908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:24.952667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:24.954270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:24.954270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:24.954270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:24.954270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:24.954270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:01:24.954270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:01:24.954270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:01:24.954270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:01:24.954270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:01:24.954270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:01:24.954270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:01:24.954270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:24.957443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 421/848] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:01:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:24.959026 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857bdb2-cb5c-4050-9049-fbc004ce68d4 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:25.003720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96160dd2-5c64-45e6-8bfd-6b995bce3a39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:25.008198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96160dd2-5c64-45e6-8bfd-6b995bce3a39 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:25.053327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:25.152108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857bdb2-cb5c-4050-9049-fbc004ce68d4 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:25.181712 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 428/849] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:01:24 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:01:25.184269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76ebf3-ae16-49ac-89b8-28eb71f5b4a1 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:25.185217 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76ebf3-ae16-49ac-89b8-28eb71f5b4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:25.185718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76ebf3-ae16-49ac-89b8-28eb71f5b4a1 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:25.188025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 429/850] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:25.189935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76ebf3-ae16-49ac-89b8-28eb71f5b4a1 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:25.214660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04d4b8-055e-4eee-9058-eaa24ad6f097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:25.215715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04d4b8-055e-4eee-9058-eaa24ad6f097 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:25.216193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04d4b8-055e-4eee-9058-eaa24ad6f097 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:25.216661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04d4b8-055e-4eee-9058-eaa24ad6f097 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:25.219000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04d4b8-055e-4eee-9058-eaa24ad6f097 None admin] RBAC: Adding query filter params (name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:25.219600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04d4b8-055e-4eee-9058-eaa24ad6f097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:25.223512 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04d4b8-055e-4eee-9058-eaa24ad6f097 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:25.245413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 422/851] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:01:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:25.248075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee84bcd-9e71-44bf-9f06-fe4a205fa558 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:25.248566 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee84bcd-9e71-44bf-9f06-fe4a205fa558 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:25.249772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee84bcd-9e71-44bf-9f06-fe4a205fa558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:25.280372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-6ee84bcd-9e71-44bf-9f06-fe4a205fa558 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:25.282480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6ee84bcd-9e71-44bf-9f06-fe4a205fa558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:25.318005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76ebf3-ae16-49ac-89b8-28eb71f5b4a1 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:25.337292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:25.340053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:25.369082 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f9259-609d-4093-a866-34ce72bacda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:25.370099 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f9259-609d-4093-a866-34ce72bacda8 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:25.370576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f9259-609d-4093-a866-34ce72bacda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:25.371051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f9259-609d-4093-a866-34ce72bacda8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:25.389452 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f9259-609d-4093-a866-34ce72bacda8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:25.392238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f9259-609d-4093-a866-34ce72bacda8 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:25.392626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 423/852] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:25.394204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee84bcd-9e71-44bf-9f06-fe4a205fa558 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-9f4f9259-609d-4093-a866-34ce72bacda8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:25.425062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:25.426917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:25.429049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:25.429049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:25.429049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:25.429049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 430/853] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:25.434497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f9259-609d-4093-a866-34ce72bacda8 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:25.583691 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee84bcd-9e71-44bf-9f06-fe4a205fa558 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:25.587707 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f9259-609d-4093-a866-34ce72bacda8 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:25.632738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6db2d-9a1f-4df9-aa0e-2caeb3e464e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:25.633413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6db2d-9a1f-4df9-aa0e-2caeb3e464e1 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:25.633695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6db2d-9a1f-4df9-aa0e-2caeb3e464e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:25.633968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6db2d-9a1f-4df9-aa0e-2caeb3e464e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:25.653505 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6db2d-9a1f-4df9-aa0e-2caeb3e464e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:25.656880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6db2d-9a1f-4df9-aa0e-2caeb3e464e1 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:25.660107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b456c8-b3bb-439b-b517-7695dd3e01a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:25.661718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b456c8-b3bb-439b-b517-7695dd3e01a1 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:25.662916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b456c8-b3bb-439b-b517-7695dd3e01a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:25.663791 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b456c8-b3bb-439b-b517-7695dd3e01a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:25.665549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b456c8-b3bb-439b-b517-7695dd3e01a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:25.666433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b456c8-b3bb-439b-b517-7695dd3e01a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:25.669670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b456c8-b3bb-439b-b517-7695dd3e01a1 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-68a6db2d-9a1f-4df9-aa0e-2caeb3e464e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:25.676619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:25.679021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:25.681428 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:25.681428 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:25.681428 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:25.681428 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 424/854] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:25.686027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6db2d-9a1f-4df9-aa0e-2caeb3e464e1 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:25.698025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 431/855] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:25.699081 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b456c8-b3bb-439b-b517-7695dd3e01a1 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:25.735892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:25.743368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:25.839536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b456c8-b3bb-439b-b517-7695dd3e01a1 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:25.895187 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8faae3b-835c-43b3-997f-f46cc5713986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:25.895894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faae3b-835c-43b3-997f-f46cc5713986 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:25.896178 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faae3b-835c-43b3-997f-f46cc5713986 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:25.896797 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faae3b-835c-43b3-997f-f46cc5713986 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:25.908202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8faae3b-835c-43b3-997f-f46cc5713986 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:25.908414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6db2d-9a1f-4df9-aa0e-2caeb3e464e1 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:25.910983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8faae3b-835c-43b3-997f-f46cc5713986 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-c8faae3b-835c-43b3-997f-f46cc5713986 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:01:25.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:25.931877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:25.932827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:01:25.932827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:01:25.932827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:01:25.932827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:01:25.932827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:01:25.932827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:01:25.932827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:25.932827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:25.933270 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 432/856] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:25.934662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8faae3b-835c-43b3-997f-f46cc5713986 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:25.978517 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b718b-b533-47f3-98d8-ffaeb6209c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:25.979304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b718b-b533-47f3-98d8-ffaeb6209c0a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:25.979798 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b718b-b533-47f3-98d8-ffaeb6209c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:25.980199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b718b-b533-47f3-98d8-ffaeb6209c0a None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:25.981345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b718b-b533-47f3-98d8-ffaeb6209c0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:25.981796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b718b-b533-47f3-98d8-ffaeb6209c0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:25.983748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b718b-b533-47f3-98d8-ffaeb6209c0a None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:26.008357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 425/857] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:26.009886 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b718b-b533-47f3-98d8-ffaeb6209c0a None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:26.088447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:26.110190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8faae3b-835c-43b3-997f-f46cc5713986 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:26.154328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:26.182922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ab884-075e-4b62-ba82-53e30adfc4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:26.183956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ab884-075e-4b62-ba82-53e30adfc4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:26.184246 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b718b-b533-47f3-98d8-ffaeb6209c0a None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:26.184440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ab884-075e-4b62-ba82-53e30adfc4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:26.184916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ab884-075e-4b62-ba82-53e30adfc4c6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:26.185858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ab884-075e-4b62-ba82-53e30adfc4c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:26.187741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ab884-075e-4b62-ba82-53e30adfc4c6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-dc5ab884-075e-4b62-ba82-53e30adfc4c6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:26.200168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:26.202039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:26.203848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 433/858] 10.222.0.61 () {56 vars in 1132 bytes} [Wed Sep 7 13:01:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:26.205473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ab884-075e-4b62-ba82-53e30adfc4c6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:26.255202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdceffb-f2d5-4f78-a66b-723180cab43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:26.256677 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdceffb-f2d5-4f78-a66b-723180cab43c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:26.257222 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdceffb-f2d5-4f78-a66b-723180cab43c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:26.257564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdceffb-f2d5-4f78-a66b-723180cab43c None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:26.258998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdceffb-f2d5-4f78-a66b-723180cab43c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:26.259561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdceffb-f2d5-4f78-a66b-723180cab43c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:26.261883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdceffb-f2d5-4f78-a66b-723180cab43c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:26.284049 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 426/859] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:26.285538 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdceffb-f2d5-4f78-a66b-723180cab43c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:26.430964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ab884-075e-4b62-ba82-53e30adfc4c6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:26.462302 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdceffb-f2d5-4f78-a66b-723180cab43c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:26.502339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c07f-379e-4f19-875f-379db2252875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:26.503402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c07f-379e-4f19-875f-379db2252875 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:26.503865 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c07f-379e-4f19-875f-379db2252875 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:26.504326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c07f-379e-4f19-875f-379db2252875 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:26.516592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce876744-a942-42e9-8e2d-9adcd225914f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:26.517143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce876744-a942-42e9-8e2d-9adcd225914f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:26.517430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce876744-a942-42e9-8e2d-9adcd225914f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:26.517676 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce876744-a942-42e9-8e2d-9adcd225914f None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:26.518340 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce876744-a942-42e9-8e2d-9adcd225914f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:26.518644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce876744-a942-42e9-8e2d-9adcd225914f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:26.519871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169c07f-379e-4f19-875f-379db2252875 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:26.520370 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce876744-a942-42e9-8e2d-9adcd225914f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:26.522670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169c07f-379e-4f19-875f-379db2252875 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:26.527227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 427/860] 10.222.0.61 () {56 vars in 1126 bytes} [Wed Sep 7 13:01:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:26.528112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce876744-a942-42e9-8e2d-9adcd225914f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:26.546227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-4169c07f-379e-4f19-875f-379db2252875 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:01:26.547362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:26.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:26.549808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:26.549808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:26.549808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:01:26.549808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:01:26.549808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:01:26.549808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:01:26.549808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:01:26.549808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:01:26.549808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:01:26.549808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:26.549808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 434/861] 10.222.0.61 () {56 vars in 1117 bytes} [Wed Sep 7 13:01:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:26.552158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107596cf-9e13-4cf5-8cba-40cc0958b1ff None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:26.553292 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107596cf-9e13-4cf5-8cba-40cc0958b1ff None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:26.553744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107596cf-9e13-4cf5-8cba-40cc0958b1ff None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:26.556247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 435/862] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:26.557798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107596cf-9e13-4cf5-8cba-40cc0958b1ff None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:26.586763 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:26.691149 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce876744-a942-42e9-8e2d-9adcd225914f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:26.738675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107596cf-9e13-4cf5-8cba-40cc0958b1ff None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:26.756628 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0da8d-713f-49de-bd4c-836a66cb5222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:26.757369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0da8d-713f-49de-bd4c-836a66cb5222 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:26.757817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0da8d-713f-49de-bd4c-836a66cb5222 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:26.758260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0da8d-713f-49de-bd4c-836a66cb5222 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:26.759529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0da8d-713f-49de-bd4c-836a66cb5222 None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=0f886e0999d34bf9870b448772cc38a3) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:26.760040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0da8d-713f-49de-bd4c-836a66cb5222 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:26.762678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0da8d-713f-49de-bd4c-836a66cb5222 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:26.772461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 428/863] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:01:26 2022] GET /identity/v3/role_assignments?user.id=0f886e0999d34bf9870b448772cc38a3&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:26.773434 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0da8d-713f-49de-bd4c-836a66cb5222 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:26.792204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f975e-5dc9-496f-9283-61dff31ed7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:26.793438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f975e-5dc9-496f-9283-61dff31ed7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:26.794027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f975e-5dc9-496f-9283-61dff31ed7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:26.794602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f975e-5dc9-496f-9283-61dff31ed7c8 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:26.796075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f975e-5dc9-496f-9283-61dff31ed7c8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:26.796662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f975e-5dc9-496f-9283-61dff31ed7c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:26.799472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f975e-5dc9-496f-9283-61dff31ed7c8 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:26.818674 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 436/864] 10.222.0.61 () {56 vars in 1126 bytes} [Wed Sep 7 13:01:26 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:26.818674 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048d44d-daf9-4d71-9a2b-20e27c47232b None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:26.818674 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048d44d-daf9-4d71-9a2b-20e27c47232b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:26.818674 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048d44d-daf9-4d71-9a2b-20e27c47232b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:26.867651 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b048d44d-daf9-4d71-9a2b-20e27c47232b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:26.870347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b048d44d-daf9-4d71-9a2b-20e27c47232b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:26.970436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:27.030705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 437/865] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:27.049437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0da8d-713f-49de-bd4c-836a66cb5222 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:27.130838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cdc85-1dbd-4db1-8ba4-2b41423baff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:27.131705 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cdc85-1dbd-4db1-8ba4-2b41423baff3 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:27.132121 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cdc85-1dbd-4db1-8ba4-2b41423baff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:27.132554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cdc85-1dbd-4db1-8ba4-2b41423baff3 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:27.133707 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cdc85-1dbd-4db1-8ba4-2b41423baff3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:27.134212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cdc85-1dbd-4db1-8ba4-2b41423baff3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:27.137197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cdc85-1dbd-4db1-8ba4-2b41423baff3 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:27.163479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 429/866] 10.222.0.61 () {56 vars in 1149 bytes} [Wed Sep 7 13:01:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:27.172093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048d44d-daf9-4d71-9a2b-20e27c47232b None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:27.186231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:27.376756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048d44d-daf9-4d71-9a2b-20e27c47232b None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:27.453949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d4dc0-6d7b-4d46-9c94-67cc5c089fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:27.454965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d4dc0-6d7b-4d46-9c94-67cc5c089fcf None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:27.455468 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d4dc0-6d7b-4d46-9c94-67cc5c089fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:27.456034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d4dc0-6d7b-4d46-9c94-67cc5c089fcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:27.483825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cdc85-1dbd-4db1-8ba4-2b41423baff3 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:27.550096 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d4dc0-6d7b-4d46-9c94-67cc5c089fcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:27.552771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d4dc0-6d7b-4d46-9c94-67cc5c089fcf None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:27.565892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:27.612672 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cdc85-1dbd-4db1-8ba4-2b41423baff3 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:27.650137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:27.657725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f774175-9eab-473a-b0c5-c77f6c2d8e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:27.658369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f774175-9eab-473a-b0c5-c77f6c2d8e85 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:27.658689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f774175-9eab-473a-b0c5-c77f6c2d8e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:27.658891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f774175-9eab-473a-b0c5-c77f6c2d8e85 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:27.659617 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f774175-9eab-473a-b0c5-c77f6c2d8e85 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:27.661065 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f774175-9eab-473a-b0c5-c77f6c2d8e85 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:27.687137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 430/867] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 13:01:27 2022] POST /identity/v3/services => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:01:27.718435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 438/868] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:01:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 13:01:27.727954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f774175-9eab-473a-b0c5-c77f6c2d8e85 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:27.875089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f774175-9eab-473a-b0c5-c77f6c2d8e85 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:27.917670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a962848-7421-42e6-927b-fe69a17e79e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:27.918259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a962848-7421-42e6-927b-fe69a17e79e7 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:27.918537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a962848-7421-42e6-927b-fe69a17e79e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:27.918901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a962848-7421-42e6-927b-fe69a17e79e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:27.937356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a962848-7421-42e6-927b-fe69a17e79e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:27.939975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a962848-7421-42e6-927b-fe69a17e79e7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-2a962848-7421-42e6-927b-fe69a17e79e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:27.968419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:27.970277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:27.971813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:27.971813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:27.971813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:27.971813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 431/869] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:27.980641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d4dc0-6d7b-4d46-9c94-67cc5c089fcf None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:28.188266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d4dc0-6d7b-4d46-9c94-67cc5c089fcf None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:28.199053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:28.230985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61212f1c-1737-47fe-855d-325f2597391c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:28.231721 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61212f1c-1737-47fe-855d-325f2597391c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:28.232002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61212f1c-1737-47fe-855d-325f2597391c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:28.232360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61212f1c-1737-47fe-855d-325f2597391c None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:28.233044 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61212f1c-1737-47fe-855d-325f2597391c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:28.233350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61212f1c-1737-47fe-855d-325f2597391c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:28.234796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61212f1c-1737-47fe-855d-325f2597391c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:28.249481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 439/870] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:28.259529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a962848-7421-42e6-927b-fe69a17e79e7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:28.310196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:28.430123 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a962848-7421-42e6-927b-fe69a17e79e7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:28.508953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ec75e-4319-485d-b9b6-d774708fb050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:28.509883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ec75e-4319-485d-b9b6-d774708fb050 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:28.510876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ec75e-4319-485d-b9b6-d774708fb050 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:28.511401 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ec75e-4319-485d-b9b6-d774708fb050 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:28.525167 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ec75e-4319-485d-b9b6-d774708fb050 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:28.528101 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ec75e-4319-485d-b9b6-d774708fb050 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-d71ec75e-4319-485d-b9b6-d774708fb050 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:01:28.552018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:28.553666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:28.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:01:28.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:01:28.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:01:28.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:01:28.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:01:28.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:01:28.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:28.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:28.555524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 432/871] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:28.564895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61212f1c-1737-47fe-855d-325f2597391c None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:28.769423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61212f1c-1737-47fe-855d-325f2597391c None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:28.852095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9769c6-d2c6-4405-8560-5cc49b6fe22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:28.853210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9769c6-d2c6-4405-8560-5cc49b6fe22e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:28.853725 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9769c6-d2c6-4405-8560-5cc49b6fe22e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:28.854226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9769c6-d2c6-4405-8560-5cc49b6fe22e None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:28.856328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9769c6-d2c6-4405-8560-5cc49b6fe22e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:28.856328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9769c6-d2c6-4405-8560-5cc49b6fe22e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:28.859166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9769c6-d2c6-4405-8560-5cc49b6fe22e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:28.886033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 440/872] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:01:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:28.895662 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ec75e-4319-485d-b9b6-d774708fb050 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:29.109516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ec75e-4319-485d-b9b6-d774708fb050 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:29.124588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:29.180438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5ea3e-ee62-495e-bcd0-52e5c93a9172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:29.181487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5ea3e-ee62-495e-bcd0-52e5c93a9172 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:29.181964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5ea3e-ee62-495e-bcd0-52e5c93a9172 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:29.182430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5ea3e-ee62-495e-bcd0-52e5c93a9172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:29.183880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5ea3e-ee62-495e-bcd0-52e5c93a9172 None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=d71863f18330414f9b6238b7e8903701) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:29.184413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5ea3e-ee62-495e-bcd0-52e5c93a9172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:29.187353 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5ea3e-ee62-495e-bcd0-52e5c93a9172 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:29.201329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 433/873] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:01:28 2022] GET /identity/v3/role_assignments?user.id=d71863f18330414f9b6238b7e8903701&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 608 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:29.985592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678decaa-9684-4453-b280-750e928df360 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:29.986340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678decaa-9684-4453-b280-750e928df360 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:29.986705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678decaa-9684-4453-b280-750e928df360 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:29.989757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 441/874] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:29.999281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487dbde-19a5-4a01-8452-b32b156eca1b None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:30.000435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487dbde-19a5-4a01-8452-b32b156eca1b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:30.000675 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487dbde-19a5-4a01-8452-b32b156eca1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:30.030422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-9487dbde-19a5-4a01-8452-b32b156eca1b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:30.032552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-9487dbde-19a5-4a01-8452-b32b156eca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:30.173370 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 434/875] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:29 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:30.283233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459118c8-c9c8-4817-99cb-44d98efafbeb None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:30.283640 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459118c8-c9c8-4817-99cb-44d98efafbeb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:30.284045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459118c8-c9c8-4817-99cb-44d98efafbeb None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:30.287803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 442/876] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:30.295830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53491f-f98d-4b2e-95dc-edd2abec6b12 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:30.296103 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53491f-f98d-4b2e-95dc-edd2abec6b12 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:30.296378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53491f-f98d-4b2e-95dc-edd2abec6b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:30.317080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8f53491f-f98d-4b2e-95dc-edd2abec6b12 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:30.318982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8f53491f-f98d-4b2e-95dc-edd2abec6b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:30.401445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 435/877] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:30 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:01:30.683550 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459118c8-c9c8-4817-99cb-44d98efafbeb None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:30.758037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:30.847597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53491f-f98d-4b2e-95dc-edd2abec6b12 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:30.876284 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:30.907137 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459118c8-c9c8-4817-99cb-44d98efafbeb None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:30.972119 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a906a-30cb-40ee-b7a5-e216d298e5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:30.972670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a906a-30cb-40ee-b7a5-e216d298e5d3 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:30.973189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a906a-30cb-40ee-b7a5-e216d298e5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:30.973486 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a906a-30cb-40ee-b7a5-e216d298e5d3 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:30.979135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a906a-30cb-40ee-b7a5-e216d298e5d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:30.979135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a906a-30cb-40ee-b7a5-e216d298e5d3 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:31.001831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 443/878] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 13:01:30 2022] POST /identity/v3/services => generated 269 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:01:31.099154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53491f-f98d-4b2e-95dc-edd2abec6b12 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:31.127793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ffeda-3fc1-47eb-af8a-4c187ba0ee98 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:31.128167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ffeda-3fc1-47eb-af8a-4c187ba0ee98 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:31.128456 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ffeda-3fc1-47eb-af8a-4c187ba0ee98 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:31.130060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 444/879] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:01:31.136520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71239337-b9b1-4c0d-b2e8-32690ebff8dd None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:31.136947 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71239337-b9b1-4c0d-b2e8-32690ebff8dd None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:31.137166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71239337-b9b1-4c0d-b2e8-32690ebff8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:31.167139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-71239337-b9b1-4c0d-b2e8-32690ebff8dd None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:31.167139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-71239337-b9b1-4c0d-b2e8-32690ebff8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:31.183150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959b5a5-cff1-42b6-ac54-53f97b2cf4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:31.183150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959b5a5-cff1-42b6-ac54-53f97b2cf4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:31.183150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959b5a5-cff1-42b6-ac54-53f97b2cf4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:31.184030 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959b5a5-cff1-42b6-ac54-53f97b2cf4f0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:31.184030 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959b5a5-cff1-42b6-ac54-53f97b2cf4f0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:31.187143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959b5a5-cff1-42b6-ac54-53f97b2cf4f0 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-e959b5a5-cff1-42b6-ac54-53f97b2cf4f0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:31.198615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:31.201704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:31.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:31.217669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 436/880] 10.222.0.61 () {56 vars in 1126 bytes} [Wed Sep 7 13:01:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:31.217669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959b5a5-cff1-42b6-ac54-53f97b2cf4f0 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:31.328048 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 445/881] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:01:31.374108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:31.399333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959b5a5-cff1-42b6-ac54-53f97b2cf4f0 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:31.444868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6ecf8-ad88-4310-af00-cb22cd0f1536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:31.445502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6ecf8-ad88-4310-af00-cb22cd0f1536 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:31.445779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6ecf8-ad88-4310-af00-cb22cd0f1536 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:31.446044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6ecf8-ad88-4310-af00-cb22cd0f1536 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:31.446685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6ecf8-ad88-4310-af00-cb22cd0f1536 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:31.446982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6ecf8-ad88-4310-af00-cb22cd0f1536 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:31.449523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6ecf8-ad88-4310-af00-cb22cd0f1536 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:31.459529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 437/882] 10.222.0.61 () {56 vars in 1122 bytes} [Wed Sep 7 13:01:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:31.464658 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71239337-b9b1-4c0d-b2e8-32690ebff8dd None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:31.497113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:31.649244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71239337-b9b1-4c0d-b2e8-32690ebff8dd None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:31.689238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37837b-8eff-4c1d-bd98-5838f19fc8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:31.689863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37837b-8eff-4c1d-bd98-5838f19fc8ee None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:31.690141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37837b-8eff-4c1d-bd98-5838f19fc8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:31.690411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37837b-8eff-4c1d-bd98-5838f19fc8ee None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:31.691154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37837b-8eff-4c1d-bd98-5838f19fc8ee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:31.691466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37837b-8eff-4c1d-bd98-5838f19fc8ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:31.693083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37837b-8eff-4c1d-bd98-5838f19fc8ee None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:31.703278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 446/883] 10.222.0.61 () {56 vars in 1122 bytes} [Wed Sep 7 13:01:31 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:31.840106 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6ecf8-ad88-4310-af00-cb22cd0f1536 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:32.036355 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6ecf8-ad88-4310-af00-cb22cd0f1536 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:32.113722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466cbad-3064-4d81-9b37-54914e306e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:32.116761 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466cbad-3064-4d81-9b37-54914e306e8c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:32.117649 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466cbad-3064-4d81-9b37-54914e306e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:32.118430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466cbad-3064-4d81-9b37-54914e306e8c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:32.119929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466cbad-3064-4d81-9b37-54914e306e8c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:32.122946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466cbad-3064-4d81-9b37-54914e306e8c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-b466cbad-3064-4d81-9b37-54914e306e8c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:32.142472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:32.144790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:01:32.146750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:32.155675 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 438/884] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:01:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:32.162482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37837b-8eff-4c1d-bd98-5838f19fc8ee None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:32.289121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:32.327505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37837b-8eff-4c1d-bd98-5838f19fc8ee None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:32.392597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976315a9-b12d-4841-a1cd-a3eaff210a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:32.393573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976315a9-b12d-4841-a1cd-a3eaff210a58 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:32.394027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976315a9-b12d-4841-a1cd-a3eaff210a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:32.394467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976315a9-b12d-4841-a1cd-a3eaff210a58 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:32.395583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976315a9-b12d-4841-a1cd-a3eaff210a58 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:32.396129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976315a9-b12d-4841-a1cd-a3eaff210a58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:32.398574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976315a9-b12d-4841-a1cd-a3eaff210a58 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:32.407685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 447/885] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:01:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:32.418683 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466cbad-3064-4d81-9b37-54914e306e8c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:32.451865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:32.628697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466cbad-3064-4d81-9b37-54914e306e8c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:32.699814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a59fd4-3448-4fbb-ad5f-3f4db3388679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:32.700891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a59fd4-3448-4fbb-ad5f-3f4db3388679 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:32.701388 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a59fd4-3448-4fbb-ad5f-3f4db3388679 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:32.701871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a59fd4-3448-4fbb-ad5f-3f4db3388679 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:32.703069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a59fd4-3448-4fbb-ad5f-3f4db3388679 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:32.703657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a59fd4-3448-4fbb-ad5f-3f4db3388679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:32.706377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a59fd4-3448-4fbb-ad5f-3f4db3388679 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:32.716758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 439/886] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:01:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:32.722870 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976315a9-b12d-4841-a1cd-a3eaff210a58 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:32.756196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a46b9b-c273-46f0-a6cb-4b99addf26ca None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:32.756720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a46b9b-c273-46f0-a6cb-4b99addf26ca None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:32.757217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a46b9b-c273-46f0-a6cb-4b99addf26ca None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:32.760623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 440/887] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:32.777780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df5fa4-0df8-45c6-a701-3d66d8811ba1 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:32.777780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df5fa4-0df8-45c6-a701-3d66d8811ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:32.777780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df5fa4-0df8-45c6-a701-3d66d8811ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:32.806482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-68df5fa4-0df8-45c6-a701-3d66d8811ba1 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:32.808740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-68df5fa4-0df8-45c6-a701-3d66d8811ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:32.900594 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976315a9-b12d-4841-a1cd-a3eaff210a58 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:32.946278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 441/888] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:01:32.971895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa80f2-9811-4e20-a5c6-b878b94c7a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:32.972542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa80f2-9811-4e20-a5c6-b878b94c7a4c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:32.972937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa80f2-9811-4e20-a5c6-b878b94c7a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:32.973388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa80f2-9811-4e20-a5c6-b878b94c7a4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:32.975254 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa80f2-9811-4e20-a5c6-b878b94c7a4c None admin] RBAC: Adding query filter params (interface=public, service_id=9d76b1ca52954927bf067a9b836d5458, region_id=RegionOne) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:32.975791 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa80f2-9811-4e20-a5c6-b878b94c7a4c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:32.978520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa80f2-9811-4e20-a5c6-b878b94c7a4c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:32.989391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 448/889] 10.222.0.61 () {56 vars in 1251 bytes} [Wed Sep 7 13:01:32 2022] GET /identity/v3/endpoints?service_id=9d76b1ca52954927bf067a9b836d5458&interface=public®ion_id=RegionOne => generated 194 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:33.001187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df5fa4-0df8-45c6-a701-3d66d8811ba1 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:33.017497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:33.212244 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df5fa4-0df8-45c6-a701-3d66d8811ba1 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:33.283256 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed4dfe-b000-44f6-8f6a-ec0ed540502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:33.284310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed4dfe-b000-44f6-8f6a-ec0ed540502e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:33.284792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed4dfe-b000-44f6-8f6a-ec0ed540502e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:33.285272 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed4dfe-b000-44f6-8f6a-ec0ed540502e None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:33.286416 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed4dfe-b000-44f6-8f6a-ec0ed540502e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:33.289168 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed4dfe-b000-44f6-8f6a-ec0ed540502e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:33.302823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 442/890] 10.222.0.61 () {56 vars in 1087 bytes} [Wed Sep 7 13:01:32 2022] GET /identity/v3/services => generated 1268 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:01:33.434862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa80f2-9811-4e20-a5c6-b878b94c7a4c None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:33.455258 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:33.642938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa80f2-9811-4e20-a5c6-b878b94c7a4c None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:33.713171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbdd66-532b-412b-9294-39837c14f12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:33.714284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbdd66-532b-412b-9294-39837c14f12b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:33.714761 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbdd66-532b-412b-9294-39837c14f12b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:33.715263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbdd66-532b-412b-9294-39837c14f12b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:33.737023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbdd66-532b-412b-9294-39837c14f12b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-b4bbdd66-532b-412b-9294-39837c14f12b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:01:33.747637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:33.748654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:33.749507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:33.749507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:33.749507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:33.749507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:33.749507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:01:33.749507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:01:33.749507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:01:33.749507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:01:33.749507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:01:33.749507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:01:33.749507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:01:33.749507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:33.750429 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 449/891] 10.222.0.61 () {56 vars in 1120 bytes} [Wed Sep 7 13:01:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:33.759794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed4dfe-b000-44f6-8f6a-ec0ed540502e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:33.894329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:33.965553 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed4dfe-b000-44f6-8f6a-ec0ed540502e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:34.038016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118b882-eb62-431d-bd63-5fd6fd341149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:34.039063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118b882-eb62-431d-bd63-5fd6fd341149 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:34.039593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118b882-eb62-431d-bd63-5fd6fd341149 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:34.040142 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118b882-eb62-431d-bd63-5fd6fd341149 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:34.041272 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118b882-eb62-431d-bd63-5fd6fd341149 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:34.041754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118b882-eb62-431d-bd63-5fd6fd341149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:34.044238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118b882-eb62-431d-bd63-5fd6fd341149 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:34.060359 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 443/892] 10.222.0.61 () {56 vars in 1115 bytes} [Wed Sep 7 13:01:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:34.071143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbdd66-532b-412b-9294-39837c14f12b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:34.121191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515f564-d4c4-438b-b606-f7dc878e2456 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:34.121639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515f564-d4c4-438b-b606-f7dc878e2456 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:34.122410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515f564-d4c4-438b-b606-f7dc878e2456 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:34.125822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 444/893] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:34.132514 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb05e00-801b-497a-91cb-17acfc4edb38 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:34.133183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb05e00-801b-497a-91cb-17acfc4edb38 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:34.133638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb05e00-801b-497a-91cb-17acfc4edb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:34.154505 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-6bb05e00-801b-497a-91cb-17acfc4edb38 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:34.156189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6bb05e00-801b-497a-91cb-17acfc4edb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:34.210227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:34.267135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 445/894] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:34.280614 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b41036-d369-494c-961d-9808347b6ed4 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:34.281065 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b41036-d369-494c-961d-9808347b6ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:34.281499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b41036-d369-494c-961d-9808347b6ed4 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:34.289106 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbdd66-532b-412b-9294-39837c14f12b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:34.291469 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 446/895] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:34 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:34.293680 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dc511-efb8-406f-9670-82edb27adf94 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:34.294365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dc511-efb8-406f-9670-82edb27adf94 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:34.295021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dc511-efb8-406f-9670-82edb27adf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:34.325212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ea7dc511-efb8-406f-9670-82edb27adf94 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:34.327498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ea7dc511-efb8-406f-9670-82edb27adf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:34.353844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4138491-361c-42f3-9cf7-7708540db2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:34.355085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4138491-361c-42f3-9cf7-7708540db2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:34.355548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4138491-361c-42f3-9cf7-7708540db2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:34.356015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4138491-361c-42f3-9cf7-7708540db2d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:34.377107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4138491-361c-42f3-9cf7-7708540db2d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:34.379771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4138491-361c-42f3-9cf7-7708540db2d3 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-e4138491-361c-42f3-9cf7-7708540db2d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:34.409412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:34.412046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:34.414062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:34.414062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:34.414062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:34.417363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 450/896] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:34.421970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4138491-361c-42f3-9cf7-7708540db2d3 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:34.444926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:34.465047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 447/897] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:34.563718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dc511-efb8-406f-9670-82edb27adf94 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:34.628101 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4138491-361c-42f3-9cf7-7708540db2d3 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:34.672353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:34.697408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904885e-61f1-4bcf-a172-a5c512b00e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:34.698799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904885e-61f1-4bcf-a172-a5c512b00e4d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:34.699538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904885e-61f1-4bcf-a172-a5c512b00e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:34.700263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904885e-61f1-4bcf-a172-a5c512b00e4d None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:34.701701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904885e-61f1-4bcf-a172-a5c512b00e4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:34.702496 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904885e-61f1-4bcf-a172-a5c512b00e4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:34.705274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904885e-61f1-4bcf-a172-a5c512b00e4d None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:34.728512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 451/898] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:34.735079 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904885e-61f1-4bcf-a172-a5c512b00e4d None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:34.766183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dc511-efb8-406f-9670-82edb27adf94 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:34.813510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165bbbf-7154-4763-a304-9bb2081f07bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:34.814163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165bbbf-7154-4763-a304-9bb2081f07bb None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:34.814444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165bbbf-7154-4763-a304-9bb2081f07bb None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:34.814725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165bbbf-7154-4763-a304-9bb2081f07bb None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:34.815435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165bbbf-7154-4763-a304-9bb2081f07bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:34.817417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165bbbf-7154-4763-a304-9bb2081f07bb None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:34.834320 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 448/899] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 13:01:34 2022] POST /identity/v3/services => generated 241 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:01:34.835230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165bbbf-7154-4763-a304-9bb2081f07bb None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:34.900571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:34.951187 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904885e-61f1-4bcf-a172-a5c512b00e4d None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:35.010289 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165bbbf-7154-4763-a304-9bb2081f07bb None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:35.026061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae604b1a-75cf-44a3-a01d-91f21c9e1750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:35.027243 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae604b1a-75cf-44a3-a01d-91f21c9e1750 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:35.027813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae604b1a-75cf-44a3-a01d-91f21c9e1750 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:35.028333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae604b1a-75cf-44a3-a01d-91f21c9e1750 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:35.042485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae604b1a-75cf-44a3-a01d-91f21c9e1750 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:35.045313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae604b1a-75cf-44a3-a01d-91f21c9e1750 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:35.049306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34715f90-3edc-44ba-bb4e-0e2ed2bea2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:35.049977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34715f90-3edc-44ba-bb4e-0e2ed2bea2ee None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:35.050255 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34715f90-3edc-44ba-bb4e-0e2ed2bea2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:35.050544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34715f90-3edc-44ba-bb4e-0e2ed2bea2ee None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:35.051184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34715f90-3edc-44ba-bb4e-0e2ed2bea2ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:35.052688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34715f90-3edc-44ba-bb4e-0e2ed2bea2ee None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-34715f90-3edc-44ba-bb4e-0e2ed2bea2ee None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:35.062790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:35.064773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:01:35.066582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:35.070525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 449/900] 10.222.0.61 () {56 vars in 1132 bytes} [Wed Sep 7 13:01:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-ae604b1a-75cf-44a3-a01d-91f21c9e1750 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:01:35.070799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:35.073639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:35.082482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:35.082482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:35.082482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:01:35.082482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:01:35.082482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:01:35.082482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:01:35.082482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:01:35.082482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:01:35.082482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:01:35.082482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:35.084968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34715f90-3edc-44ba-bb4e-0e2ed2bea2ee None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:35.085200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 452/901] 10.222.0.61 () {56 vars in 1117 bytes} [Wed Sep 7 13:01:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:35.085200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae604b1a-75cf-44a3-a01d-91f21c9e1750 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:35.168390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:35.251795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34715f90-3edc-44ba-bb4e-0e2ed2bea2ee None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:35.276231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:35.296873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae604b1a-75cf-44a3-a01d-91f21c9e1750 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:35.326551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3706228-199d-4932-8fa1-5084758a2890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:35.327734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3706228-199d-4932-8fa1-5084758a2890 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:35.328329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3706228-199d-4932-8fa1-5084758a2890 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:35.328894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3706228-199d-4932-8fa1-5084758a2890 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:35.330169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3706228-199d-4932-8fa1-5084758a2890 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:35.330836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3706228-199d-4932-8fa1-5084758a2890 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:35.334261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3706228-199d-4932-8fa1-5084758a2890 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:35.344085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b87d9-5c79-4409-afd3-1e4934373ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:35.344780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 450/902] 10.222.0.61 () {56 vars in 1126 bytes} [Wed Sep 7 13:01:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:35.345231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b87d9-5c79-4409-afd3-1e4934373ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:35.345675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b87d9-5c79-4409-afd3-1e4934373ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:35.345959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b87d9-5c79-4409-afd3-1e4934373ad4 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:35.346714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b87d9-5c79-4409-afd3-1e4934373ad4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:35.347019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b87d9-5c79-4409-afd3-1e4934373ad4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:35.348521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b87d9-5c79-4409-afd3-1e4934373ad4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:35.359278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3706228-199d-4932-8fa1-5084758a2890 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:35.370294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 453/903] 10.222.0.61 () {56 vars in 1149 bytes} [Wed Sep 7 13:01:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:35.381587 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b87d9-5c79-4409-afd3-1e4934373ad4 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:35.529348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3706228-199d-4932-8fa1-5084758a2890 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:35.570234 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1485216-26a0-47a8-a158-2968f9b8654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:35.570868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1485216-26a0-47a8-a158-2968f9b8654b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:35.571171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1485216-26a0-47a8-a158-2968f9b8654b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:35.571442 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1485216-26a0-47a8-a158-2968f9b8654b None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:35.572112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1485216-26a0-47a8-a158-2968f9b8654b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:35.572408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1485216-26a0-47a8-a158-2968f9b8654b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:35.573854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1485216-26a0-47a8-a158-2968f9b8654b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:35.584921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 451/904] 10.222.0.61 () {56 vars in 1126 bytes} [Wed Sep 7 13:01:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:35.585974 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1485216-26a0-47a8-a158-2968f9b8654b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:35.591989 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:35.602878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b87d9-5c79-4409-afd3-1e4934373ad4 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:35.633508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:35.673750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283e878-f7d9-484d-9b0b-ae56b2ed00d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:35.674544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283e878-f7d9-484d-9b0b-ae56b2ed00d0 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:35.674700 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283e878-f7d9-484d-9b0b-ae56b2ed00d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:35.674945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283e878-f7d9-484d-9b0b-ae56b2ed00d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:35.761649 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1485216-26a0-47a8-a158-2968f9b8654b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:35.762593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283e878-f7d9-484d-9b0b-ae56b2ed00d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:35.764331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283e878-f7d9-484d-9b0b-ae56b2ed00d0 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:35.841422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4556b86-fb90-4ee0-a8c9-9ab3ad30c36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:35.842912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4556b86-fb90-4ee0-a8c9-9ab3ad30c36e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:35.843796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4556b86-fb90-4ee0-a8c9-9ab3ad30c36e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:35.844595 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4556b86-fb90-4ee0-a8c9-9ab3ad30c36e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:35.846300 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4556b86-fb90-4ee0-a8c9-9ab3ad30c36e None admin] RBAC: Adding query filter params (interface=public, service_id=d5d633adf2e145c184e66122f4e54ce5, region_id=RegionOne) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:35.847230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4556b86-fb90-4ee0-a8c9-9ab3ad30c36e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:35.850691 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4556b86-fb90-4ee0-a8c9-9ab3ad30c36e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:35.875768 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 452/905] 10.222.0.61 () {56 vars in 1251 bytes} [Wed Sep 7 13:01:35 2022] GET /identity/v3/endpoints?service_id=d5d633adf2e145c184e66122f4e54ce5&interface=public®ion_id=RegionOne => generated 194 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:35.878618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4556b86-fb90-4ee0-a8c9-9ab3ad30c36e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:35.892333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:35.899676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 454/906] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:01:35 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:01:35.908951 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283e878-f7d9-484d-9b0b-ae56b2ed00d0 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:36.097458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4556b86-fb90-4ee0-a8c9-9ab3ad30c36e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:36.100143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:36.110027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283e878-f7d9-484d-9b0b-ae56b2ed00d0 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:36.157962 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0829d51-3c7e-4e7e-bab0-bf294d2e4633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:36.159048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0829d51-3c7e-4e7e-bab0-bf294d2e4633 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:36.159048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0829d51-3c7e-4e7e-bab0-bf294d2e4633 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:36.159504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0829d51-3c7e-4e7e-bab0-bf294d2e4633 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:36.160412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0829d51-3c7e-4e7e-bab0-bf294d2e4633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:36.163282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0829d51-3c7e-4e7e-bab0-bf294d2e4633 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:36.172942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fa20e-71a6-4076-ac9d-435b0390a831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:36.173942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fa20e-71a6-4076-ac9d-435b0390a831 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:36.182996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fa20e-71a6-4076-ac9d-435b0390a831 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:36.183312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 453/907] 10.222.0.61 () {56 vars in 1087 bytes} [Wed Sep 7 13:01:35 2022] GET /identity/v3/services => generated 1497 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:01:36.183477 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fa20e-71a6-4076-ac9d-435b0390a831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:36.183657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72b8d1-c062-416e-8de9-83603364b254 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:36.183657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72b8d1-c062-416e-8de9-83603364b254 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:36.183657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72b8d1-c062-416e-8de9-83603364b254 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:36.183657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 454/908] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:36.186331 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12064020-e7cd-41ab-b771-c689b42cff01 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:36.186930 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12064020-e7cd-41ab-b771-c689b42cff01 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:36.187488 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12064020-e7cd-41ab-b771-c689b42cff01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:36.204712 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fa20e-71a6-4076-ac9d-435b0390a831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:36.207576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fa20e-71a6-4076-ac9d-435b0390a831 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:36.219771 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-12064020-e7cd-41ab-b771-c689b42cff01 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:36.221959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-12064020-e7cd-41ab-b771-c689b42cff01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-f76fa20e-71a6-4076-ac9d-435b0390a831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:36.237609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:36.240302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:36.242154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:36.242154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:36.242154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:36.246567 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 455/909] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:36.247620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fa20e-71a6-4076-ac9d-435b0390a831 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:36.312520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:36.376734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fa20e-71a6-4076-ac9d-435b0390a831 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:36.382294 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 455/910] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:36.451721 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c073d-8c9f-47bc-bd09-0d455a0723d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:36.453215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c073d-8c9f-47bc-bd09-0d455a0723d9 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:36.454302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c073d-8c9f-47bc-bd09-0d455a0723d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:36.455254 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c073d-8c9f-47bc-bd09-0d455a0723d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:36.457105 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c073d-8c9f-47bc-bd09-0d455a0723d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:36.458099 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c073d-8c9f-47bc-bd09-0d455a0723d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:36.462067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c073d-8c9f-47bc-bd09-0d455a0723d9 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:36.488845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 456/911] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:36.498375 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12064020-e7cd-41ab-b771-c689b42cff01 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:36.683409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c073d-8c9f-47bc-bd09-0d455a0723d9 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:36.701360 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:36.715871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12064020-e7cd-41ab-b771-c689b42cff01 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:36.792329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f673d-550e-41f0-a225-8f50a7014f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:36.794016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f673d-550e-41f0-a225-8f50a7014f27 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:36.794989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f673d-550e-41f0-a225-8f50a7014f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:36.795786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f673d-550e-41f0-a225-8f50a7014f27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:36.810983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f673d-550e-41f0-a225-8f50a7014f27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:36.813929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f673d-550e-41f0-a225-8f50a7014f27 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-6d4f673d-550e-41f0-a225-8f50a7014f27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:01:36.839706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:36.841629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:36.843654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:01:36.843654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:01:36.843654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:01:36.843654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:01:36.843654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:01:36.843654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:01:36.843654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:36.843654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:36.847441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 456/912] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:36.857769 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f673d-550e-41f0-a225-8f50a7014f27 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:36.909399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c073d-8c9f-47bc-bd09-0d455a0723d9 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:36.932388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:36.992476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96e26c-7928-4a81-b42d-4c53fbbeb798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:36.993454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96e26c-7928-4a81-b42d-4c53fbbeb798 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:36.993944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96e26c-7928-4a81-b42d-4c53fbbeb798 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:36.994431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96e26c-7928-4a81-b42d-4c53fbbeb798 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:36.995594 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96e26c-7928-4a81-b42d-4c53fbbeb798 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:36.998116 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96e26c-7928-4a81-b42d-4c53fbbeb798 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-8a96e26c-7928-4a81-b42d-4c53fbbeb798 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:37.016328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:37.018290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:37.019959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 457/913] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:01:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:37.028860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96e26c-7928-4a81-b42d-4c53fbbeb798 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:37.095589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f673d-550e-41f0-a225-8f50a7014f27 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:37.174894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52affc-9486-4250-b187-325a9cec0477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:37.175995 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52affc-9486-4250-b187-325a9cec0477 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:37.176491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52affc-9486-4250-b187-325a9cec0477 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:37.177503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52affc-9486-4250-b187-325a9cec0477 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:37.178859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52affc-9486-4250-b187-325a9cec0477 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:37.179523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52affc-9486-4250-b187-325a9cec0477 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:37.182462 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52affc-9486-4250-b187-325a9cec0477 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:37.215576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 457/914] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:01:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:37.219873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52affc-9486-4250-b187-325a9cec0477 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:37.260688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:37.268888 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96e26c-7928-4a81-b42d-4c53fbbeb798 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:37.346511 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca67de-f608-449e-9d04-6c8bc071e1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:37.348137 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca67de-f608-449e-9d04-6c8bc071e1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:37.349562 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca67de-f608-449e-9d04-6c8bc071e1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:37.350345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca67de-f608-449e-9d04-6c8bc071e1d4 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:37.352007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca67de-f608-449e-9d04-6c8bc071e1d4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:37.352904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca67de-f608-449e-9d04-6c8bc071e1d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:37.355964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca67de-f608-449e-9d04-6c8bc071e1d4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:37.367815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 458/915] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:01:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:37.373192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca67de-f608-449e-9d04-6c8bc071e1d4 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:37.432002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52affc-9486-4250-b187-325a9cec0477 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:37.452104 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:37.476423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae456f55-bd96-4b42-8d62-7566b61b3cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:37.477090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae456f55-bd96-4b42-8d62-7566b61b3cfd None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:37.477373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae456f55-bd96-4b42-8d62-7566b61b3cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:37.477645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae456f55-bd96-4b42-8d62-7566b61b3cfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:37.478817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae456f55-bd96-4b42-8d62-7566b61b3cfd None admin] RBAC: Adding query filter params (role.id=3fe314d875214a6485d3ff844b6cebf3, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=d71863f18330414f9b6238b7e8903701) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:37.479407 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae456f55-bd96-4b42-8d62-7566b61b3cfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:37.481068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae456f55-bd96-4b42-8d62-7566b61b3cfd None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:37.487909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 458/916] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:01:37 2022] GET /identity/v3/role_assignments?user.id=d71863f18330414f9b6238b7e8903701&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=3fe314d875214a6485d3ff844b6cebf3 => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:37.596499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfacd8-bcf4-4626-9fa9-d9f711183615 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:37.596499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfacd8-bcf4-4626-9fa9-d9f711183615 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:37.596499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfacd8-bcf4-4626-9fa9-d9f711183615 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:37.596499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 459/917] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:01:37.601127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f56bb-cfc9-4144-815e-0f731b7190a0 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:37.601550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f56bb-cfc9-4144-815e-0f731b7190a0 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:37.601713 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca67de-f608-449e-9d04-6c8bc071e1d4 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:37.602023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f56bb-cfc9-4144-815e-0f731b7190a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:37.622194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-820f56bb-cfc9-4144-815e-0f731b7190a0 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:37.623716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-820f56bb-cfc9-4144-815e-0f731b7190a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:37.649576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88008758-7f75-4fd3-afdb-6e980173ca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:37.650208 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88008758-7f75-4fd3-afdb-6e980173ca15 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:37.650534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88008758-7f75-4fd3-afdb-6e980173ca15 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:37.651253 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88008758-7f75-4fd3-afdb-6e980173ca15 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:37.652003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88008758-7f75-4fd3-afdb-6e980173ca15 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:37.652320 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88008758-7f75-4fd3-afdb-6e980173ca15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:37.653783 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88008758-7f75-4fd3-afdb-6e980173ca15 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:37.665162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 459/918] 10.222.0.61 () {56 vars in 1114 bytes} [Wed Sep 7 13:01:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:37.666140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88008758-7f75-4fd3-afdb-6e980173ca15 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:37.732394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 460/919] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:01:37.800932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88008758-7f75-4fd3-afdb-6e980173ca15 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:37.804775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:37.875246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a9ec9-346d-4fea-ae68-ee60ab64e084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:37.876321 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a9ec9-346d-4fea-ae68-ee60ab64e084 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:37.876815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a9ec9-346d-4fea-ae68-ee60ab64e084 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:37.877304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a9ec9-346d-4fea-ae68-ee60ab64e084 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:37.878603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a9ec9-346d-4fea-ae68-ee60ab64e084 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:37.881849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a9ec9-346d-4fea-ae68-ee60ab64e084 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:37.943175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 460/920] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:01:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:01:38.038389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f56bb-cfc9-4144-815e-0f731b7190a0 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:38.052849 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:38.182346 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f56bb-cfc9-4144-815e-0f731b7190a0 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:38.235336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf486b-5299-44f8-8d79-7453623dfd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:38.235997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf486b-5299-44f8-8d79-7453623dfd21 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:38.236275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf486b-5299-44f8-8d79-7453623dfd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:38.236546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf486b-5299-44f8-8d79-7453623dfd21 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:38.237193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf486b-5299-44f8-8d79-7453623dfd21 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:38.238654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf486b-5299-44f8-8d79-7453623dfd21 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-8dcf486b-5299-44f8-8d79-7453623dfd21 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:38.250609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:38.252593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:38.253451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 461/921] 10.222.0.61 () {56 vars in 1126 bytes} [Wed Sep 7 13:01:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:38.261685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a9ec9-346d-4fea-ae68-ee60ab64e084 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:38.467475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a9ec9-346d-4fea-ae68-ee60ab64e084 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:38.542648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437af0a-043c-4d21-98b2-4d9fd593a53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:38.543732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437af0a-043c-4d21-98b2-4d9fd593a53a None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:38.544231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437af0a-043c-4d21-98b2-4d9fd593a53a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:38.544717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437af0a-043c-4d21-98b2-4d9fd593a53a None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:38.545899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437af0a-043c-4d21-98b2-4d9fd593a53a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:38.546445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437af0a-043c-4d21-98b2-4d9fd593a53a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:38.549152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437af0a-043c-4d21-98b2-4d9fd593a53a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:38.559262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 461/922] 10.222.0.61 () {56 vars in 1122 bytes} [Wed Sep 7 13:01:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:38.565228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf486b-5299-44f8-8d79-7453623dfd21 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:38.651693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:38.689378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf486b-5299-44f8-8d79-7453623dfd21 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:38.731330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb341766-7eb5-4f4a-baca-c1724ad8e47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:38.731880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb341766-7eb5-4f4a-baca-c1724ad8e47e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:38.732255 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb341766-7eb5-4f4a-baca-c1724ad8e47e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:38.732525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb341766-7eb5-4f4a-baca-c1724ad8e47e None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:38.733181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb341766-7eb5-4f4a-baca-c1724ad8e47e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:38.733486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb341766-7eb5-4f4a-baca-c1724ad8e47e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:38.734922 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb341766-7eb5-4f4a-baca-c1724ad8e47e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:38.741581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 462/923] 10.222.0.61 () {56 vars in 1122 bytes} [Wed Sep 7 13:01:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:38.747622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437af0a-043c-4d21-98b2-4d9fd593a53a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:38.779900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:38.936814 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437af0a-043c-4d21-98b2-4d9fd593a53a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:39.005737 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544f9d1-eb13-4cbc-8028-189d6a5b167c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:39.006822 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544f9d1-eb13-4cbc-8028-189d6a5b167c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:39.007363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544f9d1-eb13-4cbc-8028-189d6a5b167c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:39.007846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544f9d1-eb13-4cbc-8028-189d6a5b167c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:39.009232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544f9d1-eb13-4cbc-8028-189d6a5b167c None admin] RBAC: Adding query filter params (interface=public, service_id=559b010c58184efd883bf619b6ab2a05, region_id=RegionOne) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:39.009784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544f9d1-eb13-4cbc-8028-189d6a5b167c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:39.012418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544f9d1-eb13-4cbc-8028-189d6a5b167c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:39.023201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 462/924] 10.222.0.61 () {56 vars in 1251 bytes} [Wed Sep 7 13:01:38 2022] GET /identity/v3/endpoints?service_id=559b010c58184efd883bf619b6ab2a05&interface=public®ion_id=RegionOne => generated 194 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:39.032618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb341766-7eb5-4f4a-baca-c1724ad8e47e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:39.158369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb341766-7eb5-4f4a-baca-c1724ad8e47e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:39.201657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f18f7-4bc7-436a-9f4e-227bf164dbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:39.202291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f18f7-4bc7-436a-9f4e-227bf164dbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:39.202571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f18f7-4bc7-436a-9f4e-227bf164dbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:39.202842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f18f7-4bc7-436a-9f4e-227bf164dbd0 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:39.203501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f18f7-4bc7-436a-9f4e-227bf164dbd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:39.204918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f18f7-4bc7-436a-9f4e-227bf164dbd0 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:39.206385 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:39.219582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 463/925] 10.222.0.61 () {56 vars in 1087 bytes} [Wed Sep 7 13:01:39 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:01:39.669441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d354d-5428-40d7-ad78-d7c734348ff9 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:39.669745 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d354d-5428-40d7-ad78-d7c734348ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:39.670068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d354d-5428-40d7-ad78-d7c734348ff9 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:39.671679 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 463/926] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:39.677596 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425764f-fe87-4725-91d7-346990e05154 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:39.678377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425764f-fe87-4725-91d7-346990e05154 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:39.678377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425764f-fe87-4725-91d7-346990e05154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:39.698395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-7425764f-fe87-4725-91d7-346990e05154 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:39.699642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7425764f-fe87-4725-91d7-346990e05154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:39.779566 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 464/927] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:40.261903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d354d-5428-40d7-ad78-d7c734348ff9 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:40.451619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:40.475611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d354d-5428-40d7-ad78-d7c734348ff9 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:40.544681 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f388b-0650-4267-b6a9-7ec7898a829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:40.545779 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f388b-0650-4267-b6a9-7ec7898a829a None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:40.546268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f388b-0650-4267-b6a9-7ec7898a829a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:40.546746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f388b-0650-4267-b6a9-7ec7898a829a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:40.547949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f388b-0650-4267-b6a9-7ec7898a829a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:40.550507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f388b-0650-4267-b6a9-7ec7898a829a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-f62f388b-0650-4267-b6a9-7ec7898a829a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:40.569211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:40.570538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:40.572047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 464/928] 10.222.0.61 () {56 vars in 1132 bytes} [Wed Sep 7 13:01:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:40.577639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425764f-fe87-4725-91d7-346990e05154 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:40.581784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe28ea-f808-461f-bc67-550f128d2a1b None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:40.582113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe28ea-f808-461f-bc67-550f128d2a1b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:40.582355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe28ea-f808-461f-bc67-550f128d2a1b None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:40.584015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 465/929] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:01:40.589819 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae85e64-d3f8-4d39-8993-f96f040ba94b None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:40.590040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae85e64-d3f8-4d39-8993-f96f040ba94b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:40.590306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae85e64-d3f8-4d39-8993-f96f040ba94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:40.616942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-3ae85e64-d3f8-4d39-8993-f96f040ba94b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:40.619065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3ae85e64-d3f8-4d39-8993-f96f040ba94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:40.726439 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 466/930] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:40.742748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22d911-790c-45cd-9125-a2321b2443ec None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:40.743263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22d911-790c-45cd-9125-a2321b2443ec None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:40.743738 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22d911-790c-45cd-9125-a2321b2443ec None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:40.753145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 467/931] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:40 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:40.756025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98a7cf-bfbd-430a-8fa2-5dcd16295ebb None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:40.756501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98a7cf-bfbd-430a-8fa2-5dcd16295ebb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:40.756961 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98a7cf-bfbd-430a-8fa2-5dcd16295ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:40.776178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425764f-fe87-4725-91d7-346990e05154 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:40.791949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-dc98a7cf-bfbd-430a-8fa2-5dcd16295ebb None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:40.794291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-dc98a7cf-bfbd-430a-8fa2-5dcd16295ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:40.809811 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:40.819683 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f2536-ec88-4523-9b89-f4460c465653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:40.820891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f2536-ec88-4523-9b89-f4460c465653 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:40.821404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f2536-ec88-4523-9b89-f4460c465653 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:40.821883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f2536-ec88-4523-9b89-f4460c465653 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:40.822746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f2536-ec88-4523-9b89-f4460c465653 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:40.823313 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f2536-ec88-4523-9b89-f4460c465653 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:40.825174 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f2536-ec88-4523-9b89-f4460c465653 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:40.831738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 465/932] 10.222.0.61 () {56 vars in 1126 bytes} [Wed Sep 7 13:01:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:40.839175 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f2536-ec88-4523-9b89-f4460c465653 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:40.929002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 468/933] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:40.981781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f2536-ec88-4523-9b89-f4460c465653 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:41.023078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98a7cf-bfbd-430a-8fa2-5dcd16295ebb None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:41.024189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06716d3b-40fa-4051-bd33-75e21d6c86f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:41.024821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06716d3b-40fa-4051-bd33-75e21d6c86f3 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:41.025097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06716d3b-40fa-4051-bd33-75e21d6c86f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:41.025362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06716d3b-40fa-4051-bd33-75e21d6c86f3 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:41.026006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06716d3b-40fa-4051-bd33-75e21d6c86f3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:41.026304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06716d3b-40fa-4051-bd33-75e21d6c86f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:41.027979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06716d3b-40fa-4051-bd33-75e21d6c86f3 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:41.039230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 466/934] 10.222.0.61 () {56 vars in 1126 bytes} [Wed Sep 7 13:01:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:41.041527 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06716d3b-40fa-4051-bd33-75e21d6c86f3 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:41.061899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:41.125444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:41.185341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06716d3b-40fa-4051-bd33-75e21d6c86f3 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:41.231049 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854c28a-7c21-4832-913b-1e7ec25287fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:41.232044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854c28a-7c21-4832-913b-1e7ec25287fe None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:41.232469 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854c28a-7c21-4832-913b-1e7ec25287fe None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:41.232901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854c28a-7c21-4832-913b-1e7ec25287fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:41.233937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854c28a-7c21-4832-913b-1e7ec25287fe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:41.235947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854c28a-7c21-4832-913b-1e7ec25287fe None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:41.247144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98a7cf-bfbd-430a-8fa2-5dcd16295ebb None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:41.271176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 467/935] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:01:41 2022] POST /identity/v3/endpoints => generated 347 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:01:41.272920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854c28a-7c21-4832-913b-1e7ec25287fe None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:41.316871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372cb0b-7456-49a0-9723-b68d84211f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:41.317530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372cb0b-7456-49a0-9723-b68d84211f35 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:41.317980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372cb0b-7456-49a0-9723-b68d84211f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:41.318481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372cb0b-7456-49a0-9723-b68d84211f35 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:41.319500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372cb0b-7456-49a0-9723-b68d84211f35 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:41.321161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372cb0b-7456-49a0-9723-b68d84211f35 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:41.324312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: WARNING keystone.server.flask.common [None req-9372cb0b-7456-49a0-9723-b68d84211f35 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 07 13:01:41.347296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.sql.core [None req-9372cb0b-7456-49a0-9723-b68d84211f35 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 13:01:41.347296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 13:01:41.347296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [parameters: {'id': '1a5a059ab51e4a029640c29c7399ad65', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 13:01:41.347296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81796) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 13:01:41.366873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-9372cb0b-7456-49a0-9723-b68d84211f35 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1a5a059ab51e4a029640c29c7399ad65.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1a5a059ab51e4a029640c29c7399ad65. Sep 07 13:01:41.366873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:41.366873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 13:01:41.366873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 13:01:41.366873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 13:01:41.366873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 13:01:41.366873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 13:01:41.366873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 13:01:41.366873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 13:01:41.366873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application conn.query(q) Sep 07 13:01:41.366873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 13:01:41.366873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 13:01:41.366873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application result.read() Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 07 13:01:41.367990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application next(self.gen) Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application yield resource Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application next(self.gen) Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise self.value Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application session.commit() Sep 07 13:01:41.369237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self.session.flush() Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application compat.raise_( Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 13:01:41.370276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 13:01:41.371442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application conn.query(q) Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application result.read() Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 13:01:41.372510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [parameters: {'id': '1a5a059ab51e4a029640c29c7399ad65', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:41.373498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:41.374597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:41.376010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:41.376010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 07 13:01:41.376010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 07 13:01:41.376010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:41.376010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:41.376010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 07 13:01:41.376010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 07 13:01:41.376010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1a5a059ab51e4a029640c29c7399ad65. Sep 07 13:01:41.376010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:41.384620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 469/936] 10.222.0.61 () {60 vars in 1140 bytes} [Wed Sep 7 13:01:41 2022] POST /identity/v3/projects => generated 262 bytes in 362 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:41.387877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372cb0b-7456-49a0-9723-b68d84211f35 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:41.475719 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:41.501262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854c28a-7c21-4832-913b-1e7ec25287fe None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:41.523334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372cb0b-7456-49a0-9723-b68d84211f35 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:41.528261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:41.574435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe95c3f-6e9c-4ef1-a9ed-e737be4fa1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:41.574935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe95c3f-6e9c-4ef1-a9ed-e737be4fa1dc None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:41.575328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe95c3f-6e9c-4ef1-a9ed-e737be4fa1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:41.575602 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe95c3f-6e9c-4ef1-a9ed-e737be4fa1dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:41.580461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce6e80-6cc2-43d8-b322-fac084d4406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:41.581841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce6e80-6cc2-43d8-b322-fac084d4406b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:41.582574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce6e80-6cc2-43d8-b322-fac084d4406b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:41.583189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce6e80-6cc2-43d8-b322-fac084d4406b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:41.589382 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe95c3f-6e9c-4ef1-a9ed-e737be4fa1dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:01:41.593509 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce6e80-6cc2-43d8-b322-fac084d4406b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-9fe95c3f-6e9c-4ef1-a9ed-e737be4fa1dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:01:41.595765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:41.596735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:41.598141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:41.598141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:41.598141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:41.598141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:41.598141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:01:41.598141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:01:41.598141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:01:41.598141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:01:41.598141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:01:41.598141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:01:41.598141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:01:41.598141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:41.598141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 468/937] 10.222.0.61 () {56 vars in 1120 bytes} [Wed Sep 7 13:01:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:41.599659 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce6e80-6cc2-43d8-b322-fac084d4406b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:41.603539 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe95c3f-6e9c-4ef1-a9ed-e737be4fa1dc None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-4dce6e80-6cc2-43d8-b322-fac084d4406b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:01:41.617428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:41.618411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:41.619339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:01:41.619339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:01:41.619339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:01:41.619339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:01:41.619339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:01:41.619339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:01:41.619339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:41.619339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:41.623233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 470/938] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:41.628087 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce6e80-6cc2-43d8-b322-fac084d4406b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:41.738233 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe95c3f-6e9c-4ef1-a9ed-e737be4fa1dc None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:41.783548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559c42b-870d-4236-acf8-fd70d0c443d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:41.784169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559c42b-870d-4236-acf8-fd70d0c443d5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:41.784416 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559c42b-870d-4236-acf8-fd70d0c443d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:41.784654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559c42b-870d-4236-acf8-fd70d0c443d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:41.785372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559c42b-870d-4236-acf8-fd70d0c443d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:41.785677 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559c42b-870d-4236-acf8-fd70d0c443d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:41.787815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559c42b-870d-4236-acf8-fd70d0c443d5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:41.803066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 469/939] 10.222.0.61 () {56 vars in 1115 bytes} [Wed Sep 7 13:01:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:41.804165 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559c42b-870d-4236-acf8-fd70d0c443d5 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:41.827838 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce6e80-6cc2-43d8-b322-fac084d4406b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:41.846556 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:41.892520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:41.897975 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374a387-6879-4e5b-aea2-5eb6aff69f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:41.899030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374a387-6879-4e5b-aea2-5eb6aff69f29 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:41.899770 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374a387-6879-4e5b-aea2-5eb6aff69f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:41.900388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374a387-6879-4e5b-aea2-5eb6aff69f29 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:41.901940 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374a387-6879-4e5b-aea2-5eb6aff69f29 None admin] RBAC: Adding query filter params (name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:41.902690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374a387-6879-4e5b-aea2-5eb6aff69f29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:41.905810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374a387-6879-4e5b-aea2-5eb6aff69f29 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:41.926545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559c42b-870d-4236-acf8-fd70d0c443d5 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:41.955161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 471/940] 10.222.0.61 () {56 vars in 1112 bytes} [Wed Sep 7 13:01:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:41.955161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894e225-1d30-4db0-91f1-6d6e0ea86015 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:41.955161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894e225-1d30-4db0-91f1-6d6e0ea86015 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:41.955161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894e225-1d30-4db0-91f1-6d6e0ea86015 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:41.955161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 472/941] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:41.960667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9af3d8-15d2-4890-be5d-bcdcb7a54fbb None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:41.961080 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9af3d8-15d2-4890-be5d-bcdcb7a54fbb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:41.961526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9af3d8-15d2-4890-be5d-bcdcb7a54fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:41.987765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a89702-56ec-48ea-977a-5a59716bb4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:41.988906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a89702-56ec-48ea-977a-5a59716bb4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:41.990076 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a89702-56ec-48ea-977a-5a59716bb4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:41.990583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a89702-56ec-48ea-977a-5a59716bb4b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:41.993657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-aa9af3d8-15d2-4890-be5d-bcdcb7a54fbb None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:41.996503 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-aa9af3d8-15d2-4890-be5d-bcdcb7a54fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:42.006372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a89702-56ec-48ea-977a-5a59716bb4b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:42.008413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a89702-56ec-48ea-977a-5a59716bb4b6 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-c2a89702-56ec-48ea-977a-5a59716bb4b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:42.026977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:42.028132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:42.029087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:42.029087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:42.029087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:42.031159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 470/942] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:42.042319 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a89702-56ec-48ea-977a-5a59716bb4b6 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:42.159644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 473/943] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:42.208787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:42.241849 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a89702-56ec-48ea-977a-5a59716bb4b6 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:42.311036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc631b2-4d6c-4aa9-95db-c8e691e0eb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:42.312152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc631b2-4d6c-4aa9-95db-c8e691e0eb13 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:42.312640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc631b2-4d6c-4aa9-95db-c8e691e0eb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:42.313118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc631b2-4d6c-4aa9-95db-c8e691e0eb13 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:42.314348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc631b2-4d6c-4aa9-95db-c8e691e0eb13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:42.314911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc631b2-4d6c-4aa9-95db-c8e691e0eb13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:42.318417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc631b2-4d6c-4aa9-95db-c8e691e0eb13 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:42.351158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 471/944] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:42.357546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9af3d8-15d2-4890-be5d-bcdcb7a54fbb None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:42.485949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9af3d8-15d2-4890-be5d-bcdcb7a54fbb None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:42.501717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:42.527650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa9dc1-4cb3-4140-aaf0-7649817076c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:42.528308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa9dc1-4cb3-4140-aaf0-7649817076c3 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:42.528593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa9dc1-4cb3-4140-aaf0-7649817076c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:42.528872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa9dc1-4cb3-4140-aaf0-7649817076c3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:42.537817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa9dc1-4cb3-4140-aaf0-7649817076c3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:42.539344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa9dc1-4cb3-4140-aaf0-7649817076c3 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-f1fa9dc1-4cb3-4140-aaf0-7649817076c3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:01:42.553064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:42.554574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:42.556592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:42.556592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:42.556592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:01:42.556592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:01:42.556592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:01:42.556592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:01:42.556592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:01:42.556592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:01:42.556592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:01:42.556592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:42.556592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 474/945] 10.222.0.61 () {56 vars in 1117 bytes} [Wed Sep 7 13:01:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:42.566916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc631b2-4d6c-4aa9-95db-c8e691e0eb13 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:42.612529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa9dc1-4cb3-4140-aaf0-7649817076c3 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:42.742465 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc631b2-4d6c-4aa9-95db-c8e691e0eb13 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:42.773027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa9dc1-4cb3-4140-aaf0-7649817076c3 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:42.788983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e1236-9c83-4fff-91ba-e1e10913b476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:42.789783 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e1236-9c83-4fff-91ba-e1e10913b476 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:42.790099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e1236-9c83-4fff-91ba-e1e10913b476 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:42.790282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e1236-9c83-4fff-91ba-e1e10913b476 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:42.791402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e1236-9c83-4fff-91ba-e1e10913b476 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:42.791758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e1236-9c83-4fff-91ba-e1e10913b476 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:42.793417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e1236-9c83-4fff-91ba-e1e10913b476 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:42.831423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 472/946] 10.222.0.61 () {56 vars in 1149 bytes} [Wed Sep 7 13:01:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:42.833098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e1236-9c83-4fff-91ba-e1e10913b476 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:42.839494 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:42.852949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe29d0d-0ca1-4dfe-8b3a-73faa43b7309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:42.854112 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe29d0d-0ca1-4dfe-8b3a-73faa43b7309 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:42.854592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe29d0d-0ca1-4dfe-8b3a-73faa43b7309 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:42.855090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe29d0d-0ca1-4dfe-8b3a-73faa43b7309 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:42.856248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe29d0d-0ca1-4dfe-8b3a-73faa43b7309 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:42.859612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe29d0d-0ca1-4dfe-8b3a-73faa43b7309 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-efe29d0d-0ca1-4dfe-8b3a-73faa43b7309 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:42.884440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:42.885530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:42.886850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:42.887637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 475/947] 10.222.0.61 () {56 vars in 1126 bytes} [Wed Sep 7 13:01:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:42.894477 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe29d0d-0ca1-4dfe-8b3a-73faa43b7309 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:42.975140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:43.054511 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e1236-9c83-4fff-91ba-e1e10913b476 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:43.128962 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878f57c-e461-40a8-9efb-02739d5fb0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:43.130702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878f57c-e461-40a8-9efb-02739d5fb0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:43.131060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878f57c-e461-40a8-9efb-02739d5fb0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:43.131602 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878f57c-e461-40a8-9efb-02739d5fb0f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:43.135184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe29d0d-0ca1-4dfe-8b3a-73faa43b7309 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:43.155130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878f57c-e461-40a8-9efb-02739d5fb0f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:43.157477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878f57c-e461-40a8-9efb-02739d5fb0f7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-0878f57c-e461-40a8-9efb-02739d5fb0f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:43.186975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:43.188906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:43.190717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:43.190717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:43.190717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:43.195798 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 473/948] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:43.202350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878f57c-e461-40a8-9efb-02739d5fb0f7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:43.222464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634a429-078e-4751-9211-688a27c4a362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:43.224807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634a429-078e-4751-9211-688a27c4a362 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:43.225575 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634a429-078e-4751-9211-688a27c4a362 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:43.226409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634a429-078e-4751-9211-688a27c4a362 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:43.227887 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634a429-078e-4751-9211-688a27c4a362 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:43.228637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634a429-078e-4751-9211-688a27c4a362 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:43.231634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634a429-078e-4751-9211-688a27c4a362 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:43.243564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 476/949] 10.222.0.61 () {56 vars in 1122 bytes} [Wed Sep 7 13:01:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:43.251160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634a429-078e-4751-9211-688a27c4a362 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:43.320636 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:43.384802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878f57c-e461-40a8-9efb-02739d5fb0f7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:43.428453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84908aa0-6465-4a38-9fae-c837867e4d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:43.429381 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84908aa0-6465-4a38-9fae-c837867e4d10 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:43.429878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84908aa0-6465-4a38-9fae-c837867e4d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:43.430299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84908aa0-6465-4a38-9fae-c837867e4d10 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:43.431324 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84908aa0-6465-4a38-9fae-c837867e4d10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:43.431817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84908aa0-6465-4a38-9fae-c837867e4d10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:43.433734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84908aa0-6465-4a38-9fae-c837867e4d10 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:43.449358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 474/950] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:43.459525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84908aa0-6465-4a38-9fae-c837867e4d10 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:43.468437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:43.474266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634a429-078e-4751-9211-688a27c4a362 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:43.522785 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738616f3-fdf1-4a6c-aefb-2a48f5d456ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:43.523892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738616f3-fdf1-4a6c-aefb-2a48f5d456ae None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:43.524328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738616f3-fdf1-4a6c-aefb-2a48f5d456ae None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:43.524750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738616f3-fdf1-4a6c-aefb-2a48f5d456ae None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:43.525568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738616f3-fdf1-4a6c-aefb-2a48f5d456ae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:43.526059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738616f3-fdf1-4a6c-aefb-2a48f5d456ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:43.527845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738616f3-fdf1-4a6c-aefb-2a48f5d456ae None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:43.540258 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 477/951] 10.222.0.61 () {56 vars in 1122 bytes} [Wed Sep 7 13:01:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:43.541605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738616f3-fdf1-4a6c-aefb-2a48f5d456ae None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:43.662634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84908aa0-6465-4a38-9fae-c837867e4d10 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:43.669326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738616f3-fdf1-4a6c-aefb-2a48f5d456ae None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:43.698248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:43.724254 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66f0cf-c06b-4b33-b1a4-6e362bdbf488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:43.725466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37521b2-93c2-4f0c-81b4-49bd0bbb718d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:43.725629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66f0cf-c06b-4b33-b1a4-6e362bdbf488 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:43.726216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66f0cf-c06b-4b33-b1a4-6e362bdbf488 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:43.726593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37521b2-93c2-4f0c-81b4-49bd0bbb718d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:43.726877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66f0cf-c06b-4b33-b1a4-6e362bdbf488 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:43.727237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37521b2-93c2-4f0c-81b4-49bd0bbb718d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:43.727657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37521b2-93c2-4f0c-81b4-49bd0bbb718d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:43.728781 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66f0cf-c06b-4b33-b1a4-6e362bdbf488 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:43.732604 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66f0cf-c06b-4b33-b1a4-6e362bdbf488 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:43.736693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37521b2-93c2-4f0c-81b4-49bd0bbb718d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:43.738400 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37521b2-93c2-4f0c-81b4-49bd0bbb718d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-d37521b2-93c2-4f0c-81b4-49bd0bbb718d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:01:43.756194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:43.757619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:43.758784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:01:43.758784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:01:43.758784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:01:43.758784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:01:43.758784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:01:43.758784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:01:43.758784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:43.758784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:43.761679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 475/952] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:43.771345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37521b2-93c2-4f0c-81b4-49bd0bbb718d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:43.783977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 478/953] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:01:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:01:43.787384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b20d4-c531-42fd-84c5-8218a868b443 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:43.788088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b20d4-c531-42fd-84c5-8218a868b443 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:43.789424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b20d4-c531-42fd-84c5-8218a868b443 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:43.793609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 479/954] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:43.800638 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16667c7-b33c-4226-9511-144147af40e8 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:43.801279 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16667c7-b33c-4226-9511-144147af40e8 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:43.803372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16667c7-b33c-4226-9511-144147af40e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:43.850173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b16667c7-b33c-4226-9511-144147af40e8 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:43.852834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b16667c7-b33c-4226-9511-144147af40e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:43.877133 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:43.957085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 480/955] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:01:43.962092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37521b2-93c2-4f0c-81b4-49bd0bbb718d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:44.037147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db25d4-9f44-4e0c-83c7-22ee3c206ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:44.038543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db25d4-9f44-4e0c-83c7-22ee3c206ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:44.039430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db25d4-9f44-4e0c-83c7-22ee3c206ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:44.040132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db25d4-9f44-4e0c-83c7-22ee3c206ab7 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:44.041684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db25d4-9f44-4e0c-83c7-22ee3c206ab7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:44.042481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db25d4-9f44-4e0c-83c7-22ee3c206ab7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:44.045577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db25d4-9f44-4e0c-83c7-22ee3c206ab7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:44.064828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 476/956] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:44.074560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16667c7-b33c-4226-9511-144147af40e8 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:44.202890 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16667c7-b33c-4226-9511-144147af40e8 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:44.241446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db25d4-9f44-4e0c-83c7-22ee3c206ab7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:44.260893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976deea-f591-4506-a1fb-510e3058ac38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:44.261934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976deea-f591-4506-a1fb-510e3058ac38 None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:44.262223 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976deea-f591-4506-a1fb-510e3058ac38 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:44.262501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976deea-f591-4506-a1fb-510e3058ac38 None admin] PATH_INFO: `/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/d71863f18330414f9b6238b7e8903701/roles/3fe314d875214a6485d3ff844b6cebf3` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:44.265081 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976deea-f591-4506-a1fb-510e3058ac38 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user_id=d71863f18330414f9b6238b7e8903701, role_id=3fe314d875214a6485d3ff844b6cebf3)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:44.266563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976deea-f591-4506-a1fb-510e3058ac38 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:44.292037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 481/957] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:01:44 2022] PUT /identity/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/d71863f18330414f9b6238b7e8903701/roles/3fe314d875214a6485d3ff844b6cebf3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:01:44.305563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:44.458382 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db25d4-9f44-4e0c-83c7-22ee3c206ab7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:44.534580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af0e0f-2afb-4c55-9197-5752f8f6965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:44.536734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af0e0f-2afb-4c55-9197-5752f8f6965d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:44.537455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af0e0f-2afb-4c55-9197-5752f8f6965d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:44.538133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af0e0f-2afb-4c55-9197-5752f8f6965d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:44.539504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af0e0f-2afb-4c55-9197-5752f8f6965d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:44.542410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af0e0f-2afb-4c55-9197-5752f8f6965d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-58af0e0f-2afb-4c55-9197-5752f8f6965d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:44.560500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:44.562757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:01:44.564689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:44.569496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 477/958] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:44.575428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976deea-f591-4506-a1fb-510e3058ac38 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:44.601555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:44.694610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976deea-f591-4506-a1fb-510e3058ac38 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:44.735279 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3408e-c781-4a1f-998d-455b6219da39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:44.735914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3408e-c781-4a1f-998d-455b6219da39 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:44.736196 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3408e-c781-4a1f-998d-455b6219da39 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:44.736450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3408e-c781-4a1f-998d-455b6219da39 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:44.737114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3408e-c781-4a1f-998d-455b6219da39 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:44.737422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3408e-c781-4a1f-998d-455b6219da39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:44.738941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3408e-c781-4a1f-998d-455b6219da39 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:44.744907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 482/959] 10.222.0.61 () {56 vars in 1112 bytes} [Wed Sep 7 13:01:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:44.750961 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af0e0f-2afb-4c55-9197-5752f8f6965d None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:44.937945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:44.943978 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af0e0f-2afb-4c55-9197-5752f8f6965d None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:45.013955 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18303d5b-7cde-4fe6-b8f2-40d7ef073d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:45.014887 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18303d5b-7cde-4fe6-b8f2-40d7ef073d51 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:45.015443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18303d5b-7cde-4fe6-b8f2-40d7ef073d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:45.015892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18303d5b-7cde-4fe6-b8f2-40d7ef073d51 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:45.017038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18303d5b-7cde-4fe6-b8f2-40d7ef073d51 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:45.017581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18303d5b-7cde-4fe6-b8f2-40d7ef073d51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:45.019693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18303d5b-7cde-4fe6-b8f2-40d7ef073d51 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:45.029813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 478/960] 10.222.0.61 () {56 vars in 1112 bytes} [Wed Sep 7 13:01:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:45.537108 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962642ff-9fcf-486a-80e5-8c021e68bc3e None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:45.538113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962642ff-9fcf-486a-80e5-8c021e68bc3e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:45.538868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962642ff-9fcf-486a-80e5-8c021e68bc3e None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:45.541833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 483/961] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:45.550970 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcc3ad-bd9a-4841-92a5-e22ab9ff9ea2 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:45.551914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcc3ad-bd9a-4841-92a5-e22ab9ff9ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:45.552613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcc3ad-bd9a-4841-92a5-e22ab9ff9ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:45.583558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-90fcc3ad-bd9a-4841-92a5-e22ab9ff9ea2 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:45.586213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-90fcc3ad-bd9a-4841-92a5-e22ab9ff9ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:45.717372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 479/962] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:46.094374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962642ff-9fcf-486a-80e5-8c021e68bc3e None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:46.183020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:46.220530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962642ff-9fcf-486a-80e5-8c021e68bc3e None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:46.267061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd7ed9-9a25-411e-9f0c-c70167899f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:46.267727 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd7ed9-9a25-411e-9f0c-c70167899f40 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:46.268010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd7ed9-9a25-411e-9f0c-c70167899f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:46.268288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd7ed9-9a25-411e-9f0c-c70167899f40 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:46.268959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd7ed9-9a25-411e-9f0c-c70167899f40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:46.270446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd7ed9-9a25-411e-9f0c-c70167899f40 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:46.287056 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.sql.core [None req-56cd7ed9-9a25-411e-9f0c-c70167899f40 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 13:01:46.287056 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 13:01:46.287056 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [parameters: {'user_id': 'a90fe733f6804c1aad9e321a725e8c93', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 13:01:46.287056 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81796) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-56cd7ed9-9a25-411e-9f0c-c70167899f40 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application conn.query(q) Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application result.read() Sep 07 13:01:46.296794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 13:01:46.298532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application next(self.gen) Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application yield resource Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application next(self.gen) Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise self.value Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application session.commit() Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 13:01:46.301103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self.session.flush() Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application compat.raise_( Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 13:01:46.303463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 13:01:46.307046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application conn.query(q) Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application result.read() Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 13:01:46.310892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a90fe733f6804c1aad9e321a725e8c93', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:46.314148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 07 13:01:46.317190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 07 13:01:46.320889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 07 13:01:46.320889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 07 13:01:46.320889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 13:01:46.320889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 13:01:46.320889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 13:01:46.320889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:46.320889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 484/963] 10.222.0.61 () {60 vars in 1131 bytes} [Wed Sep 7 13:01:46 2022] POST /identity/v3/users => generated 160 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:46.322306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcc3ad-bd9a-4841-92a5-e22ab9ff9ea2 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:46.497604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:46.511217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcc3ad-bd9a-4841-92a5-e22ab9ff9ea2 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:46.515052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0818a1e-4edb-4e1e-a145-22835ec9d606 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:46.515393 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0818a1e-4edb-4e1e-a145-22835ec9d606 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:46.515679 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0818a1e-4edb-4e1e-a145-22835ec9d606 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:46.526440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 485/964] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:46.528248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d35a34-98c6-47f2-99b8-5100aeb348d1 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:46.528821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d35a34-98c6-47f2-99b8-5100aeb348d1 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:46.529303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d35a34-98c6-47f2-99b8-5100aeb348d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:46.549903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f2d35a34-98c6-47f2-99b8-5100aeb348d1 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:46.552432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f2d35a34-98c6-47f2-99b8-5100aeb348d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:46.578303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8d2d2-ece4-4124-bf3a-eb16eeb70eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:46.579436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8d2d2-ece4-4124-bf3a-eb16eeb70eda None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:46.579918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8d2d2-ece4-4124-bf3a-eb16eeb70eda None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:46.580373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8d2d2-ece4-4124-bf3a-eb16eeb70eda None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:46.595420 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8d2d2-ece4-4124-bf3a-eb16eeb70eda None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:46.598747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8d2d2-ece4-4124-bf3a-eb16eeb70eda None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-59a8d2d2-ece4-4124-bf3a-eb16eeb70eda None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:01:46.622774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:46.624328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:46.625323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:46.625323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:46.625323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:01:46.625323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:01:46.625323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:01:46.625323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:01:46.625323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:01:46.625323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:01:46.625323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:01:46.625323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:46.625880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 480/965] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:01:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:46.633361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8d2d2-ece4-4124-bf3a-eb16eeb70eda None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:46.646684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 486/966] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:46.799522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8d2d2-ece4-4124-bf3a-eb16eeb70eda None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:46.870351 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a1474-3d2b-426b-b541-174e3e1b4306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:46.871435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a1474-3d2b-426b-b541-174e3e1b4306 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:46.871855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a1474-3d2b-426b-b541-174e3e1b4306 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:46.872347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a1474-3d2b-426b-b541-174e3e1b4306 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:46.873675 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a1474-3d2b-426b-b541-174e3e1b4306 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:46.874479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a1474-3d2b-426b-b541-174e3e1b4306 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:46.877161 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a1474-3d2b-426b-b541-174e3e1b4306 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:46.946722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 481/967] 10.222.0.61 () {56 vars in 1101 bytes} [Wed Sep 7 13:01:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:46.959334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d35a34-98c6-47f2-99b8-5100aeb348d1 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:47.041794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:47.091299 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d35a34-98c6-47f2-99b8-5100aeb348d1 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:47.135147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac93d3-e8e3-4bdc-a80b-ad956c2fdf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:47.136025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac93d3-e8e3-4bdc-a80b-ad956c2fdf99 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:47.136318 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac93d3-e8e3-4bdc-a80b-ad956c2fdf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:47.136664 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac93d3-e8e3-4bdc-a80b-ad956c2fdf99 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:47.137298 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac93d3-e8e3-4bdc-a80b-ad956c2fdf99 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:47.138871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac93d3-e8e3-4bdc-a80b-ad956c2fdf99 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-b2ac93d3-e8e3-4bdc-a80b-ad956c2fdf99 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:47.149532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:47.152162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:47.154125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 487/968] 10.222.0.61 () {56 vars in 1126 bytes} [Wed Sep 7 13:01:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:47.160013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a1474-3d2b-426b-b541-174e3e1b4306 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:47.179589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:47.352018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a1474-3d2b-426b-b541-174e3e1b4306 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:47.429950 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea4684-9dcc-4e62-a181-352383e8464b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:47.431041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea4684-9dcc-4e62-a181-352383e8464b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:47.431601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea4684-9dcc-4e62-a181-352383e8464b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:47.432112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea4684-9dcc-4e62-a181-352383e8464b None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:47.433844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea4684-9dcc-4e62-a181-352383e8464b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:47.434408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea4684-9dcc-4e62-a181-352383e8464b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:47.437256 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea4684-9dcc-4e62-a181-352383e8464b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:47.449216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 482/969] 10.222.0.61 () {56 vars in 1122 bytes} [Wed Sep 7 13:01:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:47.455126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac93d3-e8e3-4bdc-a80b-ad956c2fdf99 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:47.569195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967a51b-cf82-4774-b807-e285c4d33638 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:47.569489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967a51b-cf82-4774-b807-e285c4d33638 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:47.569822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967a51b-cf82-4774-b807-e285c4d33638 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:47.571183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 483/970] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:01:47.579812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1846f-4ddb-40f2-9a17-99e94b00814b None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:47.580299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1846f-4ddb-40f2-9a17-99e94b00814b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:47.580728 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1846f-4ddb-40f2-9a17-99e94b00814b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:47.601593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac93d3-e8e3-4bdc-a80b-ad956c2fdf99 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:47.606247 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-0ab1846f-4ddb-40f2-9a17-99e94b00814b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:47.607418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0ab1846f-4ddb-40f2-9a17-99e94b00814b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:47.636950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:47.647477 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018d832-b1ef-46b6-b931-9bcf828e9095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:47.648112 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018d832-b1ef-46b6-b931-9bcf828e9095 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:47.648392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018d832-b1ef-46b6-b931-9bcf828e9095 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:47.648658 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018d832-b1ef-46b6-b931-9bcf828e9095 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:47.649331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018d832-b1ef-46b6-b931-9bcf828e9095 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:47.649638 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018d832-b1ef-46b6-b931-9bcf828e9095 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:47.651069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018d832-b1ef-46b6-b931-9bcf828e9095 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:47.656974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 488/971] 10.222.0.61 () {56 vars in 1122 bytes} [Wed Sep 7 13:01:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:47.667063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018d832-b1ef-46b6-b931-9bcf828e9095 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:47.731523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 484/972] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:01:47.804091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018d832-b1ef-46b6-b931-9bcf828e9095 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:47.850671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47989a2f-0856-41be-b622-e0996bc71cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:47.851593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47989a2f-0856-41be-b622-e0996bc71cac None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:47.852017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47989a2f-0856-41be-b622-e0996bc71cac None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:47.852428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47989a2f-0856-41be-b622-e0996bc71cac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:47.853334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47989a2f-0856-41be-b622-e0996bc71cac None admin] RBAC: Adding query filter params (interface=admin, service_id=559b010c58184efd883bf619b6ab2a05, region_id=RegionOne) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:47.853797 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47989a2f-0856-41be-b622-e0996bc71cac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:47.855648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47989a2f-0856-41be-b622-e0996bc71cac None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:47.862516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 489/973] 10.222.0.61 () {56 vars in 1249 bytes} [Wed Sep 7 13:01:47 2022] GET /identity/v3/endpoints?service_id=559b010c58184efd883bf619b6ab2a05&interface=admin®ion_id=RegionOne => generated 193 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:47.871964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1846f-4ddb-40f2-9a17-99e94b00814b None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:47.893259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:48.015192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d77bbf-4dad-40c5-90ba-f6a7f83c8916 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:48.015807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d77bbf-4dad-40c5-90ba-f6a7f83c8916 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:48.016419 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d77bbf-4dad-40c5-90ba-f6a7f83c8916 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:48.019278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 490/974] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:01:48.023143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d77bbf-4dad-40c5-90ba-f6a7f83c8916 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:48.038525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1846f-4ddb-40f2-9a17-99e94b00814b None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:48.079034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089df58-da27-49c3-9429-e0ec0b69c3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:48.080126 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089df58-da27-49c3-9429-e0ec0b69c3fa None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:48.080126 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089df58-da27-49c3-9429-e0ec0b69c3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:48.080436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089df58-da27-49c3-9429-e0ec0b69c3fa None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:48.080840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089df58-da27-49c3-9429-e0ec0b69c3fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:48.086682 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089df58-da27-49c3-9429-e0ec0b69c3fa None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:48.089856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 485/975] 10.222.0.61 () {56 vars in 1087 bytes} [Wed Sep 7 13:01:47 2022] GET /identity/v3/services => generated 1497 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:01:48.091191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c6463-836d-4c20-98ab-d6f2a999d68e None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:48.092808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c6463-836d-4c20-98ab-d6f2a999d68e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:48.092808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c6463-836d-4c20-98ab-d6f2a999d68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:48.112199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-fb8c6463-836d-4c20-98ab-d6f2a999d68e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:48.113542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-fb8c6463-836d-4c20-98ab-d6f2a999d68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:48.143295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:48.231140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:48.239840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 486/976] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:48.245195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d77bbf-4dad-40c5-90ba-f6a7f83c8916 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:48.292237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f93e4-3b51-49f8-89e3-61670068d564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:48.293090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f93e4-3b51-49f8-89e3-61670068d564 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:48.293496 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f93e4-3b51-49f8-89e3-61670068d564 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:48.294023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f93e4-3b51-49f8-89e3-61670068d564 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:48.308442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f93e4-3b51-49f8-89e3-61670068d564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-514f93e4-3b51-49f8-89e3-61670068d564 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:01:48.313583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:48.319706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:48.320858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:48.320858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:48.320858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:48.320858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:48.320858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:01:48.320858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:01:48.320858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:01:48.320858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:01:48.320858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:01:48.320858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:01:48.320858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:01:48.320858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:48.320858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 491/977] 10.222.0.61 () {56 vars in 1120 bytes} [Wed Sep 7 13:01:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:48.322263 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c6463-836d-4c20-98ab-d6f2a999d68e None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:48.448838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c6463-836d-4c20-98ab-d6f2a999d68e None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:48.510265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a913c01-bb3e-444f-8bef-b36ae3af1ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:48.510948 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a913c01-bb3e-444f-8bef-b36ae3af1ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:48.511354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a913c01-bb3e-444f-8bef-b36ae3af1ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:48.511623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a913c01-bb3e-444f-8bef-b36ae3af1ac2 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:48.512333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a913c01-bb3e-444f-8bef-b36ae3af1ac2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:48.512698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a913c01-bb3e-444f-8bef-b36ae3af1ac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:48.514138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a913c01-bb3e-444f-8bef-b36ae3af1ac2 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:48.525015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 487/978] 10.222.0.61 () {56 vars in 1115 bytes} [Wed Sep 7 13:01:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:48.530999 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f93e4-3b51-49f8-89e3-61670068d564 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:48.536568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a913c01-bb3e-444f-8bef-b36ae3af1ac2 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:48.653080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:48.662102 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f93e4-3b51-49f8-89e3-61670068d564 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:48.687612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:48.717246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fbc74-0278-434c-bb70-408012f63c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:48.718100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fbc74-0278-434c-bb70-408012f63c48 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:48.718502 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fbc74-0278-434c-bb70-408012f63c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:48.718907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fbc74-0278-434c-bb70-408012f63c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:48.724739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a913c01-bb3e-444f-8bef-b36ae3af1ac2 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:48.736788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fbc74-0278-434c-bb70-408012f63c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:48.739026 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fbc74-0278-434c-bb70-408012f63c48 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-767fbc74-0278-434c-bb70-408012f63c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:48.764309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:48.765351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:48.766368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:48.766368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:48.766368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:48.766368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 492/979] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:48.772936 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c13e7-72a3-4257-8fc1-4040b1669b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:48.773882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c13e7-72a3-4257-8fc1-4040b1669b72 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:48.774394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c13e7-72a3-4257-8fc1-4040b1669b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:48.774848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c13e7-72a3-4257-8fc1-4040b1669b72 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:48.774972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fbc74-0278-434c-bb70-408012f63c48 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:48.775871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c13e7-72a3-4257-8fc1-4040b1669b72 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:48.777629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c13e7-72a3-4257-8fc1-4040b1669b72 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:48.807414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 488/980] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 13:01:48 2022] POST /identity/v3/services => generated 242 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:01:48.961226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fbc74-0278-434c-bb70-408012f63c48 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:49.020045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abc81b-5d3e-497f-aa04-3650c8053629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:49.020895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abc81b-5d3e-497f-aa04-3650c8053629 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:49.021321 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abc81b-5d3e-497f-aa04-3650c8053629 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:49.021727 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abc81b-5d3e-497f-aa04-3650c8053629 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:49.022692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9abc81b-5d3e-497f-aa04-3650c8053629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:49.023186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9abc81b-5d3e-497f-aa04-3650c8053629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:49.025436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9abc81b-5d3e-497f-aa04-3650c8053629 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:49.045834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 493/981] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:49.053130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c13e7-72a3-4257-8fc1-4040b1669b72 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:49.186596 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c13e7-72a3-4257-8fc1-4040b1669b72 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:49.215537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:49.234684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254359f-b6fc-479b-8341-fe696fdb61dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:49.235632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254359f-b6fc-479b-8341-fe696fdb61dc None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:49.235929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254359f-b6fc-479b-8341-fe696fdb61dc None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:49.236224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254359f-b6fc-479b-8341-fe696fdb61dc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:49.246806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254359f-b6fc-479b-8341-fe696fdb61dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:49.248326 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254359f-b6fc-479b-8341-fe696fdb61dc None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-b254359f-b6fc-479b-8341-fe696fdb61dc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:01:49.262645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:49.264921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:49.266749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:49.266749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:49.266749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:01:49.266749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:01:49.266749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:01:49.266749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:01:49.266749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:01:49.266749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:01:49.266749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:01:49.266749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:49.266749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 489/982] 10.222.0.61 () {56 vars in 1117 bytes} [Wed Sep 7 13:01:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:49.277348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abc81b-5d3e-497f-aa04-3650c8053629 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:49.310672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:49.415592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abc81b-5d3e-497f-aa04-3650c8053629 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:49.458251 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8ebba-abf5-4e4d-9deb-a1ae82ba66b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:49.458885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8ebba-abf5-4e4d-9deb-a1ae82ba66b6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:49.459218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8ebba-abf5-4e4d-9deb-a1ae82ba66b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:49.459462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8ebba-abf5-4e4d-9deb-a1ae82ba66b6 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:49.460231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8ebba-abf5-4e4d-9deb-a1ae82ba66b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:49.460541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8ebba-abf5-4e4d-9deb-a1ae82ba66b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:49.462080 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8ebba-abf5-4e4d-9deb-a1ae82ba66b6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:49.484978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 494/983] 10.222.0.61 () {56 vars in 1149 bytes} [Wed Sep 7 13:01:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:49.495293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254359f-b6fc-479b-8341-fe696fdb61dc None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:49.690010 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254359f-b6fc-479b-8341-fe696fdb61dc None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:49.732949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdb631-77e6-499d-bddb-e1ff944387b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:49.733691 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdb631-77e6-499d-bddb-e1ff944387b6 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:49.733972 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdb631-77e6-499d-bddb-e1ff944387b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:49.734244 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdb631-77e6-499d-bddb-e1ff944387b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:49.792999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:49.813887 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdb631-77e6-499d-bddb-e1ff944387b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:49.815389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdb631-77e6-499d-bddb-e1ff944387b6 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:49.927126 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 490/984] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:01:49 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:01:49.932781 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8ebba-abf5-4e4d-9deb-a1ae82ba66b6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:50.029699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:50.068650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8ebba-abf5-4e4d-9deb-a1ae82ba66b6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:50.141154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270c35a-a946-426d-8ea0-9fbb2d74ebb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:50.142229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270c35a-a946-426d-8ea0-9fbb2d74ebb8 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:50.142788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270c35a-a946-426d-8ea0-9fbb2d74ebb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:50.143309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270c35a-a946-426d-8ea0-9fbb2d74ebb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:50.164377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270c35a-a946-426d-8ea0-9fbb2d74ebb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:50.167078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270c35a-a946-426d-8ea0-9fbb2d74ebb8 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-f270c35a-a946-426d-8ea0-9fbb2d74ebb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:01:50.198796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:50.207397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:01:50.209374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:01:50.209374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:01:50.209374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:50.209374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 495/985] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:50.213926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdb631-77e6-499d-bddb-e1ff944387b6 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:50.374619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:50.414405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdb631-77e6-499d-bddb-e1ff944387b6 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:50.483263 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c26df-d171-4672-a519-32565f300e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:50.484321 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c26df-d171-4672-a519-32565f300e27 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:50.484789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c26df-d171-4672-a519-32565f300e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:50.485284 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c26df-d171-4672-a519-32565f300e27 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:50.486457 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c26df-d171-4672-a519-32565f300e27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:50.486968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c26df-d171-4672-a519-32565f300e27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:50.489581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c26df-d171-4672-a519-32565f300e27 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:50.515167 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 491/986] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:50.518073 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270c35a-a946-426d-8ea0-9fbb2d74ebb8 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:50.651805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270c35a-a946-426d-8ea0-9fbb2d74ebb8 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:50.689248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:50.696746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61897c3a-6696-482c-b8bc-cd95254949db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:50.697412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897c3a-6696-482c-b8bc-cd95254949db None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:50.697692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897c3a-6696-482c-b8bc-cd95254949db None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:50.698022 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897c3a-6696-482c-b8bc-cd95254949db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:50.706233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61897c3a-6696-482c-b8bc-cd95254949db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:50.708350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61897c3a-6696-482c-b8bc-cd95254949db None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-61897c3a-6696-482c-b8bc-cd95254949db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:01:50.721937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:50.724014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:50.726006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:01:50.726006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:01:50.726006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:01:50.726006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:01:50.726006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:01:50.726006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:01:50.726006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:01:50.726006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:50.731039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 496/987] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:50.731378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c26df-d171-4672-a519-32565f300e27 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:50.929155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c26df-d171-4672-a519-32565f300e27 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:50.990984 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd04cc-fb22-44ec-808b-3ac4cd7d7731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:50.991756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd04cc-fb22-44ec-808b-3ac4cd7d7731 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:50.992034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd04cc-fb22-44ec-808b-3ac4cd7d7731 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:50.992298 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd04cc-fb22-44ec-808b-3ac4cd7d7731 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:50.993034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd04cc-fb22-44ec-808b-3ac4cd7d7731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:50.993475 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd04cc-fb22-44ec-808b-3ac4cd7d7731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:50.995693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd04cc-fb22-44ec-808b-3ac4cd7d7731 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:51.014824 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 492/988] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:01:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:51.022654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61897c3a-6696-482c-b8bc-cd95254949db None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:51.105014 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fa587-9a9e-4df7-9b26-6909d38532a5 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:51.105467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fa587-9a9e-4df7-9b26-6909d38532a5 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:51.105910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fa587-9a9e-4df7-9b26-6909d38532a5 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:51.108479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 493/989] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:01:51.117663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fec045-3059-4421-99c6-963e0c3ac4fa None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:51.118125 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fec045-3059-4421-99c6-963e0c3ac4fa None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:51.118565 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fec045-3059-4421-99c6-963e0c3ac4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:51.129025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:51.147741 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-d7fec045-3059-4421-99c6-963e0c3ac4fa None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:51.149735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d7fec045-3059-4421-99c6-963e0c3ac4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:51.221763 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61897c3a-6696-482c-b8bc-cd95254949db None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:51.259932 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 494/990] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:51.286330 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e20d7-ee9e-4e27-8918-da615655b34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:51.287239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e20d7-ee9e-4e27-8918-da615655b34b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:51.287666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e20d7-ee9e-4e27-8918-da615655b34b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:51.288090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e20d7-ee9e-4e27-8918-da615655b34b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:51.288984 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e20d7-ee9e-4e27-8918-da615655b34b None admin] RBAC: Adding query filter params (role.id=3fe314d875214a6485d3ff844b6cebf3, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=d71863f18330414f9b6238b7e8903701) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:51.289428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e20d7-ee9e-4e27-8918-da615655b34b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:51.291054 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e20d7-ee9e-4e27-8918-da615655b34b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:51.307140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 497/991] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:01:51 2022] GET /identity/v3/role_assignments?user.id=d71863f18330414f9b6238b7e8903701&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=3fe314d875214a6485d3ff844b6cebf3 => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:51.541901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fec045-3059-4421-99c6-963e0c3ac4fa None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:51.651241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fec045-3059-4421-99c6-963e0c3ac4fa None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:51.684976 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:51.692055 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d0a24-5925-49a5-a824-e0acd20814fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:51.693052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0a24-5925-49a5-a824-e0acd20814fb None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:51.693560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0a24-5925-49a5-a824-e0acd20814fb None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:51.693560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0a24-5925-49a5-a824-e0acd20814fb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:51.693973 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d0a24-5925-49a5-a824-e0acd20814fb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:51.695494 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d0a24-5925-49a5-a824-e0acd20814fb None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-c22d0a24-5925-49a5-a824-e0acd20814fb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:51.705720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:51.706795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:01:51.708405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:51.709455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 495/992] 10.222.0.61 () {56 vars in 1126 bytes} [Wed Sep 7 13:01:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:01:51.713289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e20d7-ee9e-4e27-8918-da615655b34b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:51.781135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:51.833682 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e20d7-ee9e-4e27-8918-da615655b34b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:51.871993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ecae6-73ea-44bd-84c8-5b7a4079d06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:51.872634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ecae6-73ea-44bd-84c8-5b7a4079d06a None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:51.872918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ecae6-73ea-44bd-84c8-5b7a4079d06a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:51.873197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ecae6-73ea-44bd-84c8-5b7a4079d06a None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:51.874123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ecae6-73ea-44bd-84c8-5b7a4079d06a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:51.874432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ecae6-73ea-44bd-84c8-5b7a4079d06a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:51.875895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ecae6-73ea-44bd-84c8-5b7a4079d06a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:51.878647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e3e87-5efa-42e8-a49d-5f3815a17d40 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:51.879347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e3e87-5efa-42e8-a49d-5f3815a17d40 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:51.879347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e3e87-5efa-42e8-a49d-5f3815a17d40 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:51.883184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 496/993] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:51.883462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 498/994] 10.222.0.61 () {56 vars in 1122 bytes} [Wed Sep 7 13:01:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:51.888190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ecae6-73ea-44bd-84c8-5b7a4079d06a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:51.889487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e7a37-3d05-4046-8e68-1c11e620f40e None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:51.890048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e7a37-3d05-4046-8e68-1c11e620f40e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:51.890575 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e7a37-3d05-4046-8e68-1c11e620f40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:51.920955 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8a4e7a37-3d05-4046-8e68-1c11e620f40e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:51.923003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8a4e7a37-3d05-4046-8e68-1c11e620f40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:52.022318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 497/995] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:52.034409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ecae6-73ea-44bd-84c8-5b7a4079d06a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:52.072201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:52.097471 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816afb0-f055-4d73-aa63-de4048fcb61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:52.098967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816afb0-f055-4d73-aa63-de4048fcb61a None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:52.099460 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816afb0-f055-4d73-aa63-de4048fcb61a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:52.099999 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816afb0-f055-4d73-aa63-de4048fcb61a None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:52.100923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816afb0-f055-4d73-aa63-de4048fcb61a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:52.101388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816afb0-f055-4d73-aa63-de4048fcb61a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:52.102963 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816afb0-f055-4d73-aa63-de4048fcb61a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:52.110211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 499/996] 10.222.0.61 () {56 vars in 1122 bytes} [Wed Sep 7 13:01:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:52.120117 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e7a37-3d05-4046-8e68-1c11e620f40e None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:52.302468 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816afb0-f055-4d73-aa63-de4048fcb61a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:52.302669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e7a37-3d05-4046-8e68-1c11e620f40e None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:52.337460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:52.350188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a271590-b6e5-438d-a081-272d2a4c53a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:52.350807 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a271590-b6e5-438d-a081-272d2a4c53a9 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:52.351088 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a271590-b6e5-438d-a081-272d2a4c53a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:52.351474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a271590-b6e5-438d-a081-272d2a4c53a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:52.352089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a271590-b6e5-438d-a081-272d2a4c53a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:52.353698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a271590-b6e5-438d-a081-272d2a4c53a9 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:52.373015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 498/997] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:01:52 2022] POST /identity/v3/endpoints => generated 325 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:01:52.502782 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816afb0-f055-4d73-aa63-de4048fcb61a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:52.583798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9dbfb-54b7-4e22-94ba-0d17b088b927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:52.584913 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9dbfb-54b7-4e22-94ba-0d17b088b927 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:52.585398 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9dbfb-54b7-4e22-94ba-0d17b088b927 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:52.586573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9dbfb-54b7-4e22-94ba-0d17b088b927 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:52.586961 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9dbfb-54b7-4e22-94ba-0d17b088b927 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:52.589585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9dbfb-54b7-4e22-94ba-0d17b088b927 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-24c9dbfb-54b7-4e22-94ba-0d17b088b927 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:52.607645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:52.608720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:01:52.609581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:01:52.610351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 500/998] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:52.617118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a271590-b6e5-438d-a081-272d2a4c53a9 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:52.816186 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a271590-b6e5-438d-a081-272d2a4c53a9 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:52.881554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09615c9-b551-43bb-a80a-1fe137637cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:52.882524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09615c9-b551-43bb-a80a-1fe137637cba None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:52.883011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09615c9-b551-43bb-a80a-1fe137637cba None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:52.883502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09615c9-b551-43bb-a80a-1fe137637cba None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:52.884654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09615c9-b551-43bb-a80a-1fe137637cba None admin] RBAC: Adding query filter params (name=compute) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:52.885196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09615c9-b551-43bb-a80a-1fe137637cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:52.887737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09615c9-b551-43bb-a80a-1fe137637cba None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:52.901786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 499/999] 10.222.0.61 () {56 vars in 1112 bytes} [Wed Sep 7 13:01:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:52.903742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9dbfb-54b7-4e22-94ba-0d17b088b927 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:52.991011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:53.092139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9dbfb-54b7-4e22-94ba-0d17b088b927 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:53.134237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedaed92-3d0b-407c-a4f0-37f248f79a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:53.134746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedaed92-3d0b-407c-a4f0-37f248f79a70 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:53.135029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedaed92-3d0b-407c-a4f0-37f248f79a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:53.135341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedaed92-3d0b-407c-a4f0-37f248f79a70 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:53.136127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedaed92-3d0b-407c-a4f0-37f248f79a70 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:53.136432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedaed92-3d0b-407c-a4f0-37f248f79a70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:53.137816 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedaed92-3d0b-407c-a4f0-37f248f79a70 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:53.144104 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 501/1000] 10.222.0.61 () {56 vars in 1112 bytes} [Wed Sep 7 13:01:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:53.151215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09615c9-b551-43bb-a80a-1fe137637cba None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:53.248967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:53.348419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09615c9-b551-43bb-a80a-1fe137637cba None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:53.403402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ab390-61d1-491d-9379-a49a98b5f0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:53.403932 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ab390-61d1-491d-9379-a49a98b5f0bd None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:53.404184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ab390-61d1-491d-9379-a49a98b5f0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:53.404447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ab390-61d1-491d-9379-a49a98b5f0bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:53.405091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ab390-61d1-491d-9379-a49a98b5f0bd None admin] RBAC: Adding query filter params (interface=public, service_id=9ba581fa59dc47d09a3370fac51377d2, region_id=RegionOne) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:53.405395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ab390-61d1-491d-9379-a49a98b5f0bd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:53.406773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ab390-61d1-491d-9379-a49a98b5f0bd None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:53.413170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 500/1001] 10.222.0.61 () {56 vars in 1251 bytes} [Wed Sep 7 13:01:53 2022] GET /identity/v3/endpoints?service_id=9ba581fa59dc47d09a3370fac51377d2&interface=public®ion_id=RegionOne => generated 194 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:53.419954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedaed92-3d0b-407c-a4f0-37f248f79a70 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:53.607723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedaed92-3d0b-407c-a4f0-37f248f79a70 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:53.625980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:53.674343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f3232-7773-44bb-a3b3-83d4c745e38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:53.675331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f3232-7773-44bb-a3b3-83d4c745e38c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:53.675991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f3232-7773-44bb-a3b3-83d4c745e38c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:53.676283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f3232-7773-44bb-a3b3-83d4c745e38c None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:53.677083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f3232-7773-44bb-a3b3-83d4c745e38c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:53.679729 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f3232-7773-44bb-a3b3-83d4c745e38c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:53.699483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 502/1002] 10.222.0.61 () {56 vars in 1087 bytes} [Wed Sep 7 13:01:53 2022] GET /identity/v3/services => generated 1727 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:01:54.768779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7d43f-7dbb-405a-80e4-a0595e0cb411 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:54.770270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7d43f-7dbb-405a-80e4-a0595e0cb411 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:54.770270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7d43f-7dbb-405a-80e4-a0595e0cb411 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:54.775164 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 501/1003] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:01:54.780474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498facd-f5c1-4555-9e5d-7bd0f20f2b27 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:54.780994 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498facd-f5c1-4555-9e5d-7bd0f20f2b27 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:54.781426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498facd-f5c1-4555-9e5d-7bd0f20f2b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:54.812871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-7498facd-f5c1-4555-9e5d-7bd0f20f2b27 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:54.814909 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7498facd-f5c1-4555-9e5d-7bd0f20f2b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:54.918959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 503/1004] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:54 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:55.429311 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7d43f-7dbb-405a-80e4-a0595e0cb411 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:55.467472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8538ae-da51-4dd3-91ea-79f4a67d0754 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:55.467923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8538ae-da51-4dd3-91ea-79f4a67d0754 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:55.468404 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8538ae-da51-4dd3-91ea-79f4a67d0754 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:55.470672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 504/1005] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:01:55.480038 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda0695-1514-4387-83b1-aaff4e721dac None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:55.480615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda0695-1514-4387-83b1-aaff4e721dac None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:55.481123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda0695-1514-4387-83b1-aaff4e721dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:55.510245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-2dda0695-1514-4387-83b1-aaff4e721dac None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:55.512020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2dda0695-1514-4387-83b1-aaff4e721dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:55.627123 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7d43f-7dbb-405a-80e4-a0595e0cb411 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:55.637717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 505/1006] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:55 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:01:55.654972 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:55.705979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260506bd-b36e-4ed8-8c26-b71589d0e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:55.707619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260506bd-b36e-4ed8-8c26-b71589d0e345 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:55.707992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260506bd-b36e-4ed8-8c26-b71589d0e345 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:55.708461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260506bd-b36e-4ed8-8c26-b71589d0e345 None admin] PATH_INFO: `/v3/services/image` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:55.709520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260506bd-b36e-4ed8-8c26-b71589d0e345 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:55.712068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260506bd-b36e-4ed8-8c26-b71589d0e345 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-260506bd-b36e-4ed8-8c26-b71589d0e345 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:55.731452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:55.735506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:55.737329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 502/1007] 10.222.0.61 () {56 vars in 1105 bytes} [Wed Sep 7 13:01:55 2022] GET /identity/v3/services/image => generated 86 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:55.743955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda0695-1514-4387-83b1-aaff4e721dac None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:55.801945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:55.944088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda0695-1514-4387-83b1-aaff4e721dac None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:56.014392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260506bd-b36e-4ed8-8c26-b71589d0e345 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:56.015973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf802f-50dc-4772-b467-e866e6478660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:56.016740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf802f-50dc-4772-b467-e866e6478660 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:56.017203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf802f-50dc-4772-b467-e866e6478660 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:56.017612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf802f-50dc-4772-b467-e866e6478660 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:56.018443 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf802f-50dc-4772-b467-e866e6478660 None admin] RBAC: Adding query filter params (name=image) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:56.018874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf802f-50dc-4772-b467-e866e6478660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:56.021218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf802f-50dc-4772-b467-e866e6478660 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:56.028162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 506/1008] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:56.034945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf802f-50dc-4772-b467-e866e6478660 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:56.170882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:56.179303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf802f-50dc-4772-b467-e866e6478660 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:56.179933 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260506bd-b36e-4ed8-8c26-b71589d0e345 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:56.221952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be0a1a-8a45-41e9-82ae-a1e65dd50683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:56.223514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be0a1a-8a45-41e9-82ae-a1e65dd50683 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:56.224015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be0a1a-8a45-41e9-82ae-a1e65dd50683 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:56.224556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be0a1a-8a45-41e9-82ae-a1e65dd50683 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:56.225740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be0a1a-8a45-41e9-82ae-a1e65dd50683 None admin] RBAC: Adding query filter params (type=image) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:56.226420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be0a1a-8a45-41e9-82ae-a1e65dd50683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:56.228969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d3533-7cd5-4acf-86ed-f3305b865c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:56.229157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be0a1a-8a45-41e9-82ae-a1e65dd50683 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:56.229611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d3533-7cd5-4acf-86ed-f3305b865c8b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:56.229883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d3533-7cd5-4acf-86ed-f3305b865c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:56.230199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d3533-7cd5-4acf-86ed-f3305b865c8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:56.231594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d3533-7cd5-4acf-86ed-f3305b865c8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:56.233129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d3533-7cd5-4acf-86ed-f3305b865c8b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:56.234810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 503/1009] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:56.236061 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcaf23-f052-4ed0-8345-99925a5c13f2 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:56.236352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcaf23-f052-4ed0-8345-99925a5c13f2 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:56.236589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcaf23-f052-4ed0-8345-99925a5c13f2 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:56.237768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 507/1010] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:01:56.237949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 504/1011] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:56.244985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be0a1a-8a45-41e9-82ae-a1e65dd50683 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:56.247921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb1842-66dd-4e4f-aca2-ce0a7ccb0c4e None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:56.248424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb1842-66dd-4e4f-aca2-ce0a7ccb0c4e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:56.249176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb1842-66dd-4e4f-aca2-ce0a7ccb0c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:56.282526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-35eb1842-66dd-4e4f-aca2-ce0a7ccb0c4e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:56.284846 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-35eb1842-66dd-4e4f-aca2-ce0a7ccb0c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:56.360703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:56.398509 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be0a1a-8a45-41e9-82ae-a1e65dd50683 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:56.399514 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 505/1012] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:56 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:56.462794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae435c8-b096-4742-a566-2d1c808aefd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:56.463859 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae435c8-b096-4742-a566-2d1c808aefd6 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:56.464307 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae435c8-b096-4742-a566-2d1c808aefd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:56.464751 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae435c8-b096-4742-a566-2d1c808aefd6 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:56.465874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae435c8-b096-4742-a566-2d1c808aefd6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:56.468579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae435c8-b096-4742-a566-2d1c808aefd6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:56.513956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 508/1013] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 13:01:56 2022] POST /identity/v3/projects => generated 311 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:01:56.671978 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb1842-66dd-4e4f-aca2-ce0a7ccb0c4e None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:56.823601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb1842-66dd-4e4f-aca2-ce0a7ccb0c4e None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:56.866357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c7596-b722-4266-8938-0af88fd5c323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:56.867033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c7596-b722-4266-8938-0af88fd5c323 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:56.867348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c7596-b722-4266-8938-0af88fd5c323 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:56.867618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c7596-b722-4266-8938-0af88fd5c323 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:56.868234 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c7596-b722-4266-8938-0af88fd5c323 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:56.869659 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c7596-b722-4266-8938-0af88fd5c323 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-0d0c7596-b722-4266-8938-0af88fd5c323 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:01:56.880436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:01:56.881524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:01:56.882391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 506/1014] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:01:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:01:56.888229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae435c8-b096-4742-a566-2d1c808aefd6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:56.911259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:57.015268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae435c8-b096-4742-a566-2d1c808aefd6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:57.054263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b8403-33c1-4f7d-b90a-0c373f1a6d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:57.054907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b8403-33c1-4f7d-b90a-0c373f1a6d9c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:57.055222 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b8403-33c1-4f7d-b90a-0c373f1a6d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:57.055507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b8403-33c1-4f7d-b90a-0c373f1a6d9c None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:57.056184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b8403-33c1-4f7d-b90a-0c373f1a6d9c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:57.056500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b8403-33c1-4f7d-b90a-0c373f1a6d9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:57.057980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b8403-33c1-4f7d-b90a-0c373f1a6d9c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:57.069185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 509/1015] 10.222.0.61 () {56 vars in 1112 bytes} [Wed Sep 7 13:01:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:57.070725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c7596-b722-4266-8938-0af88fd5c323 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:57.131314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:57.207412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c7596-b722-4266-8938-0af88fd5c323 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:57.249615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185162e-aa97-44ac-a277-83b380cff13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:57.250307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185162e-aa97-44ac-a277-83b380cff13d None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:57.250585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185162e-aa97-44ac-a277-83b380cff13d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:57.250852 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185162e-aa97-44ac-a277-83b380cff13d None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:57.251534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185162e-aa97-44ac-a277-83b380cff13d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:01:57.251836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185162e-aa97-44ac-a277-83b380cff13d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:57.253246 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185162e-aa97-44ac-a277-83b380cff13d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:57.264198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 507/1016] 10.222.0.61 () {56 vars in 1112 bytes} [Wed Sep 7 13:01:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:57.266172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b8403-33c1-4f7d-b90a-0c373f1a6d9c None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:57.380768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:57.425587 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b8403-33c1-4f7d-b90a-0c373f1a6d9c None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:57.492171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17e62d-6685-446d-94d2-4d14689b1936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:57.493247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17e62d-6685-446d-94d2-4d14689b1936 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:57.493762 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17e62d-6685-446d-94d2-4d14689b1936 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:57.494246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17e62d-6685-446d-94d2-4d14689b1936 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:57.495574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17e62d-6685-446d-94d2-4d14689b1936 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:57.498117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17e62d-6685-446d-94d2-4d14689b1936 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:57.530367 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 510/1017] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:01:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:01:58.957673 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31c3eb-a6e9-40c4-89a7-c288f16217b7 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:58.958520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31c3eb-a6e9-40c4-89a7-c288f16217b7 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:58.958520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31c3eb-a6e9-40c4-89a7-c288f16217b7 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:58.960562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 508/1018] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:01:58.966816 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ef929-f10c-4306-a43d-bcbbc5e401fe None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:58.967689 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ef929-f10c-4306-a43d-bcbbc5e401fe None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:58.967849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ef929-f10c-4306-a43d-bcbbc5e401fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:58.990485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-fe2ef929-f10c-4306-a43d-bcbbc5e401fe None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:01:58.991817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-fe2ef929-f10c-4306-a43d-bcbbc5e401fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:01:59.099176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 511/1019] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:01:58 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:01:59.380202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31c3eb-a6e9-40c4-89a7-c288f16217b7 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:59.514395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31c3eb-a6e9-40c4-89a7-c288f16217b7 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:59.517418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:59.561521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7460d52-dbde-4ed8-9345-7202a407ac6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:59.562183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7460d52-dbde-4ed8-9345-7202a407ac6b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:59.562461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7460d52-dbde-4ed8-9345-7202a407ac6b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:59.562730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7460d52-dbde-4ed8-9345-7202a407ac6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:59.564515 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7460d52-dbde-4ed8-9345-7202a407ac6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:59.566151 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7460d52-dbde-4ed8-9345-7202a407ac6b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:59.568231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 509/1020] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:01:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:01:59.575051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ef929-f10c-4306-a43d-bcbbc5e401fe None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:01:59.686808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:01:59.767492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ef929-f10c-4306-a43d-bcbbc5e401fe None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:01:59.838078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38110996-cf16-48b9-91d1-bfac60ab7228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:01:59.839160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38110996-cf16-48b9-91d1-bfac60ab7228 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:59.839646 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38110996-cf16-48b9-91d1-bfac60ab7228 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:59.840144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38110996-cf16-48b9-91d1-bfac60ab7228 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:59.841374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38110996-cf16-48b9-91d1-bfac60ab7228 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:01:59.844094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38110996-cf16-48b9-91d1-bfac60ab7228 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:01:59.887148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 512/1021] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 13:01:59 2022] POST /identity/v3/users => generated 285 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:01:59.995699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9009bd-151f-43ae-8af0-77374b3ce6e2 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:01:59.996487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9009bd-151f-43ae-8af0-77374b3ce6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:01:59.996646 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9009bd-151f-43ae-8af0-77374b3ce6e2 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:01:59.999292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 510/1022] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:01:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:00.009031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c6424-0e14-4e9d-bfbc-25e0f7fc76c3 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:00.009795 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c6424-0e14-4e9d-bfbc-25e0f7fc76c3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:00.010147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c6424-0e14-4e9d-bfbc-25e0f7fc76c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:00.022999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0f00d-0597-46ae-a836-d525b6d6be5c None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:00.023308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0f00d-0597-46ae-a836-d525b6d6be5c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:00.023593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0f00d-0597-46ae-a836-d525b6d6be5c None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:00.025034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 511/1023] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:02:00.031378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c23bce-c6a4-4eb5-aa7f-9ad4960d406c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:00.031702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c23bce-c6a4-4eb5-aa7f-9ad4960d406c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:00.031926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c23bce-c6a4-4eb5-aa7f-9ad4960d406c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:00.041136 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-101c6424-0e14-4e9d-bfbc-25e0f7fc76c3 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:00.042597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-101c6424-0e14-4e9d-bfbc-25e0f7fc76c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:00.062377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-a8c23bce-c6a4-4eb5-aa7f-9ad4960d406c None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:00.064372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a8c23bce-c6a4-4eb5-aa7f-9ad4960d406c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:00.163404 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 513/1024] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:00 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:00.173437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 512/1025] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:00 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:02:00.455873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c6424-0e14-4e9d-bfbc-25e0f7fc76c3 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:00.513412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:00.586720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c6424-0e14-4e9d-bfbc-25e0f7fc76c3 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:00.633200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7d02c-2b08-4383-91b9-246aebd5e627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:00.634073 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7d02c-2b08-4383-91b9-246aebd5e627 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:00.634496 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7d02c-2b08-4383-91b9-246aebd5e627 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:00.634904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7d02c-2b08-4383-91b9-246aebd5e627 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:00.648639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7d02c-2b08-4383-91b9-246aebd5e627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-c4a7d02c-2b08-4383-91b9-246aebd5e627 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:02:00.653793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:00.655675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:00.657942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:00.657942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:00.657942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:00.657942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:00.657942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:02:00.657942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:02:00.657942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:02:00.657942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:02:00.657942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:02:00.657942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:02:00.657942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:02:00.657942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:02:00.665764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 514/1026] 10.222.0.61 () {56 vars in 1120 bytes} [Wed Sep 7 13:02:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:02:00.666035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c23bce-c6a4-4eb5-aa7f-9ad4960d406c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:00.681511 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7d02c-2b08-4383-91b9-246aebd5e627 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:00.827333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c23bce-c6a4-4eb5-aa7f-9ad4960d406c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:00.829636 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:00.867695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7d02c-2b08-4383-91b9-246aebd5e627 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:00.872850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:00.886069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10c028-5b8b-474e-9de6-3b29e6b4459f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:00.887144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10c028-5b8b-474e-9de6-3b29e6b4459f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:00.887596 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10c028-5b8b-474e-9de6-3b29e6b4459f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:00.888066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10c028-5b8b-474e-9de6-3b29e6b4459f None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:00.889305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10c028-5b8b-474e-9de6-3b29e6b4459f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:00.889840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10c028-5b8b-474e-9de6-3b29e6b4459f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:00.892516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10c028-5b8b-474e-9de6-3b29e6b4459f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:00.906792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 513/1027] 10.222.0.61 () {56 vars in 1115 bytes} [Wed Sep 7 13:02:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:00.917486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10c028-5b8b-474e-9de6-3b29e6b4459f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:00.936287 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68921e-407b-479d-b5ad-945d5f664007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:00.937336 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68921e-407b-479d-b5ad-945d5f664007 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:00.937837 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68921e-407b-479d-b5ad-945d5f664007 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:00.938329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68921e-407b-479d-b5ad-945d5f664007 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:00.939555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68921e-407b-479d-b5ad-945d5f664007 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:00.941866 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68921e-407b-479d-b5ad-945d5f664007 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:00.964115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 515/1028] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 13:02:00 2022] POST /identity/v3/services => generated 242 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:02:01.145037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10c028-5b8b-474e-9de6-3b29e6b4459f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:01.227918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70beba-205a-4e01-a1de-35187fd67337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:01.228925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70beba-205a-4e01-a1de-35187fd67337 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:01.229371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70beba-205a-4e01-a1de-35187fd67337 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:01.229825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70beba-205a-4e01-a1de-35187fd67337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:01.251821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70beba-205a-4e01-a1de-35187fd67337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:01.254904 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70beba-205a-4e01-a1de-35187fd67337 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-ed70beba-205a-4e01-a1de-35187fd67337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:02:01.282421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:01.285104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:02:01.287266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:02:01.287266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:01.287266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:01.287266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 514/1029] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:01.294010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68921e-407b-479d-b5ad-945d5f664007 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:01.448538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68921e-407b-479d-b5ad-945d5f664007 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:01.492408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c584b-f6b5-41ea-a184-d9038020eed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:01.493046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c584b-f6b5-41ea-a184-d9038020eed0 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:01.493327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c584b-f6b5-41ea-a184-d9038020eed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:01.493596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c584b-f6b5-41ea-a184-d9038020eed0 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:01.494300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c584b-f6b5-41ea-a184-d9038020eed0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:01.494699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c584b-f6b5-41ea-a184-d9038020eed0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:01.497363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c584b-f6b5-41ea-a184-d9038020eed0 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:01.511168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 516/1030] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:01.518333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70beba-205a-4e01-a1de-35187fd67337 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:01.540776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:01.651464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70beba-205a-4e01-a1de-35187fd67337 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:01.695331 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f47358-d553-49ca-b4f7-a9097d52ab5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:01.695979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f47358-d553-49ca-b4f7-a9097d52ab5c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:01.696341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f47358-d553-49ca-b4f7-a9097d52ab5c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:01.696611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f47358-d553-49ca-b4f7-a9097d52ab5c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:01.707507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f47358-d553-49ca-b4f7-a9097d52ab5c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:01.708879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f47358-d553-49ca-b4f7-a9097d52ab5c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-73f47358-d553-49ca-b4f7-a9097d52ab5c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:02:01.723394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:01.725304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:01.727303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:01.727303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:01.727303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:02:01.727303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:02:01.727303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:02:01.727303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:02:01.727303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:02:01.727303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:02:01.727303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:02:01.727303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:01.727303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 515/1031] 10.222.0.61 () {56 vars in 1093 bytes} [Wed Sep 7 13:02:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:02:01.734145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c584b-f6b5-41ea-a184-d9038020eed0 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:01.783669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:01.894083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c584b-f6b5-41ea-a184-d9038020eed0 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:01.938700 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f62bd9-11c0-4f89-8ca8-13bfd76edcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:01.939351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f62bd9-11c0-4f89-8ca8-13bfd76edcc4 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:01.939633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f62bd9-11c0-4f89-8ca8-13bfd76edcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:01.939902 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f62bd9-11c0-4f89-8ca8-13bfd76edcc4 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:01.940657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f62bd9-11c0-4f89-8ca8-13bfd76edcc4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:01.941525 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f62bd9-11c0-4f89-8ca8-13bfd76edcc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:01.942977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f62bd9-11c0-4f89-8ca8-13bfd76edcc4 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:01.966413 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 517/1032] 10.222.0.61 () {56 vars in 1133 bytes} [Wed Sep 7 13:02:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:01.973403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f47358-d553-49ca-b4f7-a9097d52ab5c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:02.120297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:02.184097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f47358-d553-49ca-b4f7-a9097d52ab5c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:02.251662 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf17b6-3c23-40b5-a1e5-8e5b363ae353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:02.252610 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf17b6-3c23-40b5-a1e5-8e5b363ae353 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:02.253095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf17b6-3c23-40b5-a1e5-8e5b363ae353 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:02.254126 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf17b6-3c23-40b5-a1e5-8e5b363ae353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:02.388487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf17b6-3c23-40b5-a1e5-8e5b363ae353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:02.391513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf17b6-3c23-40b5-a1e5-8e5b363ae353 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:02.525279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:02.593781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 516/1033] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:02:01 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:02:02.596241 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f62bd9-11c0-4f89-8ca8-13bfd76edcc4 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:02.764140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:02.783823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7149913-02f1-41d0-8090-173c991ecb25 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:02.784199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7149913-02f1-41d0-8090-173c991ecb25 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:02.785323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7149913-02f1-41d0-8090-173c991ecb25 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:02.791471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 517/1034] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:02.793769 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cb4f7-5c97-4859-a4fc-24affa370874 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:02.794053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cb4f7-5c97-4859-a4fc-24affa370874 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:02.794467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cb4f7-5c97-4859-a4fc-24affa370874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:02.805792 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f62bd9-11c0-4f89-8ca8-13bfd76edcc4 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:02.815299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ae4cb4f7-5c97-4859-a4fc-24affa370874 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:02.816528 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ae4cb4f7-5c97-4859-a4fc-24affa370874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:02.874700 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404d135-cf11-4937-ae16-aee6ba179674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:02.875822 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404d135-cf11-4937-ae16-aee6ba179674 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:02.876322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404d135-cf11-4937-ae16-aee6ba179674 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:02.876797 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404d135-cf11-4937-ae16-aee6ba179674 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:02.896370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404d135-cf11-4937-ae16-aee6ba179674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:02.899327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404d135-cf11-4937-ae16-aee6ba179674 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:02.909949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 518/1035] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-f404d135-cf11-4937-ae16-aee6ba179674 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:02:02.928896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:02.930731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:02:02.933590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:02:02.933590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:02.933590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:02:02.933590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 518/1036] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:02.940568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cb4f7-5c97-4859-a4fc-24affa370874 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:03.085716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cb4f7-5c97-4859-a4fc-24affa370874 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:03.093969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:03.129812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b6c34-c751-4a82-a658-2044c3e07957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:03.130442 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b6c34-c751-4a82-a658-2044c3e07957 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:03.130719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b6c34-c751-4a82-a658-2044c3e07957 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:03.130981 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b6c34-c751-4a82-a658-2044c3e07957 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:03.131681 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b6c34-c751-4a82-a658-2044c3e07957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:03.131982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b6c34-c751-4a82-a658-2044c3e07957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:03.133584 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b6c34-c751-4a82-a658-2044c3e07957 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:03.155144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 519/1037] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:03.158061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404d135-cf11-4937-ae16-aee6ba179674 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:03.296495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b6c34-c751-4a82-a658-2044c3e07957 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:03.346769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404d135-cf11-4937-ae16-aee6ba179674 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:03.389452 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05294e-6069-468a-8eb1-6e1635e51cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:03.390110 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05294e-6069-468a-8eb1-6e1635e51cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:03.390393 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05294e-6069-468a-8eb1-6e1635e51cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:03.390657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05294e-6069-468a-8eb1-6e1635e51cc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:03.393113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:03.400142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05294e-6069-468a-8eb1-6e1635e51cc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:03.402256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05294e-6069-468a-8eb1-6e1635e51cc5 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-1e05294e-6069-468a-8eb1-6e1635e51cc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:02:03.416203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:03.425937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:03.427862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:02:03.427862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:02:03.427862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:02:03.427862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:02:03.427862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:02:03.427862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:02:03.427862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:02:03.427862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:02:03.427862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 519/1038] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:02:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:02:03.427862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05294e-6069-468a-8eb1-6e1635e51cc5 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:03.514547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b6c34-c751-4a82-a658-2044c3e07957 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:03.585674 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246b5a3-6963-4ed7-ab3d-99b170d7cd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:03.586913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246b5a3-6963-4ed7-ab3d-99b170d7cd13 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:03.587469 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246b5a3-6963-4ed7-ab3d-99b170d7cd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:03.588067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246b5a3-6963-4ed7-ab3d-99b170d7cd13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:03.602714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05294e-6069-468a-8eb1-6e1635e51cc5 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:03.608285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246b5a3-6963-4ed7-ab3d-99b170d7cd13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-1246b5a3-6963-4ed7-ab3d-99b170d7cd13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:02:03.621850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:03.624305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:03.626345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:03.626345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:03.626345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:03.626345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:03.626345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:02:03.626345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:02:03.626345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:02:03.626345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:02:03.626345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:02:03.626345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:02:03.626345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:02:03.626345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:03.626345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 520/1039] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:02:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:02:03.632249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246b5a3-6963-4ed7-ab3d-99b170d7cd13 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:03.653085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd174fc-90f2-4aad-986c-98b60af28b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:03.653085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd174fc-90f2-4aad-986c-98b60af28b53 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:03.653085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd174fc-90f2-4aad-986c-98b60af28b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:03.653085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd174fc-90f2-4aad-986c-98b60af28b53 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:03.653791 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd174fc-90f2-4aad-986c-98b60af28b53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:03.653925 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd174fc-90f2-4aad-986c-98b60af28b53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:03.655477 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd174fc-90f2-4aad-986c-98b60af28b53 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:03.688672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 520/1040] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:02:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:03.688937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:03.690342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd174fc-90f2-4aad-986c-98b60af28b53 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:03.791673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:03.828474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246b5a3-6963-4ed7-ab3d-99b170d7cd13 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:03.876040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb943505-5614-44cf-a33d-a0dad7e1080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:03.876633 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb943505-5614-44cf-a33d-a0dad7e1080b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:03.876911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb943505-5614-44cf-a33d-a0dad7e1080b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:03.877178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb943505-5614-44cf-a33d-a0dad7e1080b None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:03.877862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb943505-5614-44cf-a33d-a0dad7e1080b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:03.878189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb943505-5614-44cf-a33d-a0dad7e1080b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:03.879815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb943505-5614-44cf-a33d-a0dad7e1080b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:03.887475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd174fc-90f2-4aad-986c-98b60af28b53 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:03.890395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 521/1041] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:02:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:03.896760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb943505-5614-44cf-a33d-a0dad7e1080b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:03.955523 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54935bbb-1a99-4b8b-8d70-67e4e90bf1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:03.957208 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54935bbb-1a99-4b8b-8d70-67e4e90bf1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:03.957634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54935bbb-1a99-4b8b-8d70-67e4e90bf1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:03.958068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54935bbb-1a99-4b8b-8d70-67e4e90bf1a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:03.959306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54935bbb-1a99-4b8b-8d70-67e4e90bf1a0 None admin] RBAC: Adding query filter params (role.id=3fe314d875214a6485d3ff844b6cebf3, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=ba93e2d4aaed4c5595d5fd7363626bcb) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:03.959786 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54935bbb-1a99-4b8b-8d70-67e4e90bf1a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:03.965016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54935bbb-1a99-4b8b-8d70-67e4e90bf1a0 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:03.987142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 521/1042] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:02:03 2022] GET /identity/v3/role_assignments?user.id=ba93e2d4aaed4c5595d5fd7363626bcb&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=3fe314d875214a6485d3ff844b6cebf3 => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:04.074409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:04.114838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb943505-5614-44cf-a33d-a0dad7e1080b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:04.193552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3267ef-a53e-4aa1-a4e2-cf60ba2542ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:04.194650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3267ef-a53e-4aa1-a4e2-cf60ba2542ea None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:04.195175 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3267ef-a53e-4aa1-a4e2-cf60ba2542ea None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:04.195684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3267ef-a53e-4aa1-a4e2-cf60ba2542ea None admin] PATH_INFO: `/v3/users/dee0dca30734456da33c7b2ec0534ebe` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:04.226146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3267ef-a53e-4aa1-a4e2-cf60ba2542ea None admin] RBAC: Authorizing `identity:get_user(user_id=dee0dca30734456da33c7b2ec0534ebe)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:04.229274 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3267ef-a53e-4aa1-a4e2-cf60ba2542ea None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:04.234114 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 522/1043] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:02:03 2022] GET /identity/v3/users/dee0dca30734456da33c7b2ec0534ebe => generated 285 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:04.243631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54935bbb-1a99-4b8b-8d70-67e4e90bf1a0 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:04.327177 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2070f68-d6ed-40f6-9a17-c2482147fc92 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:04.328219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2070f68-d6ed-40f6-9a17-c2482147fc92 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:04.328711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2070f68-d6ed-40f6-9a17-c2482147fc92 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:04.330827 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 523/1044] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:04.337759 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787b2e1-9f79-418e-9956-696d24d51a71 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:04.338417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787b2e1-9f79-418e-9956-696d24d51a71 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:04.338911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787b2e1-9f79-418e-9956-696d24d51a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:04.362053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-a787b2e1-9f79-418e-9956-696d24d51a71 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:04.363983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a787b2e1-9f79-418e-9956-696d24d51a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:04.432005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54935bbb-1a99-4b8b-8d70-67e4e90bf1a0 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:04.458619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:04.470052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 524/1045] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:04.494202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed90f76-9bec-4bd6-a4cd-f9eec21f0c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:04.495245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed90f76-9bec-4bd6-a4cd-f9eec21f0c26 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:04.495685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed90f76-9bec-4bd6-a4cd-f9eec21f0c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:04.496123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed90f76-9bec-4bd6-a4cd-f9eec21f0c26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:04.635362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed90f76-9bec-4bd6-a4cd-f9eec21f0c26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:04.638129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed90f76-9bec-4bd6-a4cd-f9eec21f0c26 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:04.762579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787b2e1-9f79-418e-9956-696d24d51a71 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:04.814247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 522/1046] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:02:04 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:02:04.821418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed90f76-9bec-4bd6-a4cd-f9eec21f0c26 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:04.845382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:04.908176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:04.966104 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed90f76-9bec-4bd6-a4cd-f9eec21f0c26 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:04.975778 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787b2e1-9f79-418e-9956-696d24d51a71 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:05.016857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782f0ed-45ac-4b24-a17c-07048e518385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:05.017736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782f0ed-45ac-4b24-a17c-07048e518385 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:05.018201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782f0ed-45ac-4b24-a17c-07048e518385 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:05.018702 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782f0ed-45ac-4b24-a17c-07048e518385 None admin] PATH_INFO: `/v3/projects/81e2949d0aea446998480e9f18835ee1` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:05.020279 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782f0ed-45ac-4b24-a17c-07048e518385 None admin] RBAC: Authorizing `identity:get_project(project_id=81e2949d0aea446998480e9f18835ee1)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:05.022240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782f0ed-45ac-4b24-a17c-07048e518385 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:05.030340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 523/1047] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:02:04 2022] GET /identity/v3/projects/81e2949d0aea446998480e9f18835ee1 => generated 311 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:05.032050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782f0ed-45ac-4b24-a17c-07048e518385 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:05.046404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa35d5-a91c-4199-92de-9322bf6dc4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:05.047383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa35d5-a91c-4199-92de-9322bf6dc4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:05.047830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa35d5-a91c-4199-92de-9322bf6dc4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:05.048286 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa35d5-a91c-4199-92de-9322bf6dc4f2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:05.049317 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa35d5-a91c-4199-92de-9322bf6dc4f2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:05.051739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa35d5-a91c-4199-92de-9322bf6dc4f2 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-6aaa35d5-a91c-4199-92de-9322bf6dc4f2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:05.063557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:05.066392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:05.067606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 525/1048] 10.222.0.61 () {56 vars in 1105 bytes} [Wed Sep 7 13:02:04 2022] GET /identity/v3/services/image => generated 86 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:05.077283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa35d5-a91c-4199-92de-9322bf6dc4f2 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:05.192256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:05.214215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782f0ed-45ac-4b24-a17c-07048e518385 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:05.261729 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542b076-8000-48cc-bf77-ae0d2c20a8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:05.262797 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542b076-8000-48cc-bf77-ae0d2c20a8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:05.263257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542b076-8000-48cc-bf77-ae0d2c20a8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:05.263668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542b076-8000-48cc-bf77-ae0d2c20a8e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:05.265061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542b076-8000-48cc-bf77-ae0d2c20a8e0 None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=81e2949d0aea446998480e9f18835ee1, user.id=dee0dca30734456da33c7b2ec0534ebe) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:05.265545 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542b076-8000-48cc-bf77-ae0d2c20a8e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:05.267714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542b076-8000-48cc-bf77-ae0d2c20a8e0 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:05.276431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 524/1049] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:02:05 2022] GET /identity/v3/role_assignments?user.id=dee0dca30734456da33c7b2ec0534ebe&scope.project.id=81e2949d0aea446998480e9f18835ee1&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:05.277265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa35d5-a91c-4199-92de-9322bf6dc4f2 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:05.350283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e09b4-1175-4ad7-8d5b-d1548dcd7bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:05.351473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e09b4-1175-4ad7-8d5b-d1548dcd7bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:05.354013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e09b4-1175-4ad7-8d5b-d1548dcd7bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:05.354013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e09b4-1175-4ad7-8d5b-d1548dcd7bd3 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:05.354013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e09b4-1175-4ad7-8d5b-d1548dcd7bd3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:05.354013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e09b4-1175-4ad7-8d5b-d1548dcd7bd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:05.356194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e09b4-1175-4ad7-8d5b-d1548dcd7bd3 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:05.360883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:05.371446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 526/1050] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:05.378714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542b076-8000-48cc-bf77-ae0d2c20a8e0 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:05.543514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542b076-8000-48cc-bf77-ae0d2c20a8e0 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:05.617098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08f493-9fa9-4a00-b09a-85b40786e3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:05.618186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08f493-9fa9-4a00-b09a-85b40786e3be None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:05.618691 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08f493-9fa9-4a00-b09a-85b40786e3be None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:05.619239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08f493-9fa9-4a00-b09a-85b40786e3be None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:05.620442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08f493-9fa9-4a00-b09a-85b40786e3be None admin] RBAC: Adding query filter params (type=image) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:05.621005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08f493-9fa9-4a00-b09a-85b40786e3be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:05.624011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08f493-9fa9-4a00-b09a-85b40786e3be None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:05.635173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 525/1051] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:05.642024 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e09b4-1175-4ad7-8d5b-d1548dcd7bd3 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:05.830075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e09b4-1175-4ad7-8d5b-d1548dcd7bd3 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:05.894594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1c310-c386-4902-bbc9-b8fd5277973f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:05.895673 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1c310-c386-4902-bbc9-b8fd5277973f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:05.896170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1c310-c386-4902-bbc9-b8fd5277973f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:05.896664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1c310-c386-4902-bbc9-b8fd5277973f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:05.898054 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1c310-c386-4902-bbc9-b8fd5277973f None admin] RBAC: Adding query filter params (interface=public, service_id=97b33f9256a440f784a3d2a0628c51ac, region_id=RegionOne) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:05.898638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1c310-c386-4902-bbc9-b8fd5277973f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:05.901208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1c310-c386-4902-bbc9-b8fd5277973f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:05.916069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 527/1052] 10.222.0.61 () {56 vars in 1251 bytes} [Wed Sep 7 13:02:05 2022] GET /identity/v3/endpoints?service_id=97b33f9256a440f784a3d2a0628c51ac&interface=public®ion_id=RegionOne => generated 194 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:05.917994 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08f493-9fa9-4a00-b09a-85b40786e3be None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:05.950084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:06.104869 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08f493-9fa9-4a00-b09a-85b40786e3be None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:06.148784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfc1a8-c365-45bc-a31b-db39071e64b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:06.149409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfc1a8-c365-45bc-a31b-db39071e64b5 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:06.149715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfc1a8-c365-45bc-a31b-db39071e64b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:06.149963 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfc1a8-c365-45bc-a31b-db39071e64b5 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:06.150703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfc1a8-c365-45bc-a31b-db39071e64b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:06.152212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfc1a8-c365-45bc-a31b-db39071e64b5 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:06.160322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 526/1053] 10.222.0.61 () {56 vars in 1087 bytes} [Wed Sep 7 13:02:05 2022] GET /identity/v3/services => generated 1957 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:02:06.387291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4fbbd-2247-4e65-b309-bfea6ed894df None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:06.388041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4fbbd-2247-4e65-b309-bfea6ed894df None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:06.388041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4fbbd-2247-4e65-b309-bfea6ed894df None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:06.389316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 528/1054] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:06.396357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2d799-2893-44ea-9cf8-feb9f100b688 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:06.396841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2d799-2893-44ea-9cf8-feb9f100b688 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:06.397316 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2d799-2893-44ea-9cf8-feb9f100b688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:06.429139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-cbb2d799-2893-44ea-9cf8-feb9f100b688 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:06.431253 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-cbb2d799-2893-44ea-9cf8-feb9f100b688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:06.567443 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 527/1055] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:06.913292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4fbbd-2247-4e65-b309-bfea6ed894df None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:06.992177 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:07.046765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4fbbd-2247-4e65-b309-bfea6ed894df None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:07.113163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f14bfe-4cd0-4f4d-bdd4-2d56196802a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:07.114251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f14bfe-4cd0-4f4d-bdd4-2d56196802a3 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:07.114718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f14bfe-4cd0-4f4d-bdd4-2d56196802a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:07.115216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f14bfe-4cd0-4f4d-bdd4-2d56196802a3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:07.137767 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f14bfe-4cd0-4f4d-bdd4-2d56196802a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-b6f14bfe-4cd0-4f4d-bdd4-2d56196802a3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:02:07.146403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:07.147704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:07.151599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:07.151599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:07.151599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:07.151599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:07.151599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:02:07.151599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:02:07.151599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:02:07.151599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:02:07.151599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:02:07.151599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:02:07.151599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:02:07.151599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:07.151599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 529/1056] 10.222.0.61 () {56 vars in 1120 bytes} [Wed Sep 7 13:02:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:07.155199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2d799-2893-44ea-9cf8-feb9f100b688 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:07.168971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:07.293531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2d799-2893-44ea-9cf8-feb9f100b688 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:07.339445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3b54c-f73a-4b60-b799-cc8c95fdb3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:07.340043 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3b54c-f73a-4b60-b799-cc8c95fdb3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:07.340299 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3b54c-f73a-4b60-b799-cc8c95fdb3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:07.340578 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3b54c-f73a-4b60-b799-cc8c95fdb3e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:07.341325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3b54c-f73a-4b60-b799-cc8c95fdb3e6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:07.341702 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3b54c-f73a-4b60-b799-cc8c95fdb3e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:07.343194 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3b54c-f73a-4b60-b799-cc8c95fdb3e6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:07.354742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 528/1057] 10.222.0.61 () {56 vars in 1115 bytes} [Wed Sep 7 13:02:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:07.361197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f14bfe-4cd0-4f4d-bdd4-2d56196802a3 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:07.519594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f14bfe-4cd0-4f4d-bdd4-2d56196802a3 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:07.551758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:07.567269 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e5236-012b-46f0-9d20-dd0dd189e6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:07.567919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e5236-012b-46f0-9d20-dd0dd189e6eb None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:07.568185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e5236-012b-46f0-9d20-dd0dd189e6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:07.568453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e5236-012b-46f0-9d20-dd0dd189e6eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:07.582908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e5236-012b-46f0-9d20-dd0dd189e6eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:07.584443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e5236-012b-46f0-9d20-dd0dd189e6eb None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-8e3e5236-012b-46f0-9d20-dd0dd189e6eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:02:07.601502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:07.614178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:02:07.616257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:02:07.616257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:07.616257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:07.616257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 530/1058] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:07.616728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3b54c-f73a-4b60-b799-cc8c95fdb3e6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:07.755316 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:07.821550 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3b54c-f73a-4b60-b799-cc8c95fdb3e6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:07.891644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3512f40-ff0d-42c6-908b-f12310c71bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:07.892736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3512f40-ff0d-42c6-908b-f12310c71bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:07.893237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3512f40-ff0d-42c6-908b-f12310c71bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:07.894246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3512f40-ff0d-42c6-908b-f12310c71bc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:07.895502 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3512f40-ff0d-42c6-908b-f12310c71bc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:07.896048 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3512f40-ff0d-42c6-908b-f12310c71bc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:07.898632 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3512f40-ff0d-42c6-908b-f12310c71bc1 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:07.927776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 529/1059] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:07.929769 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e5236-012b-46f0-9d20-dd0dd189e6eb None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:07.947967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fc25f-2992-4075-bbff-c6a460e7b1e7 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:07.948343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fc25f-2992-4075-bbff-c6a460e7b1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:07.948536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fc25f-2992-4075-bbff-c6a460e7b1e7 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:07.950170 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 530/1060] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:07.955651 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e882c-80b4-4ca9-b2f0-af2a0fd9c429 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:07.956468 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e882c-80b4-4ca9-b2f0-af2a0fd9c429 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:07.956748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e882c-80b4-4ca9-b2f0-af2a0fd9c429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:07.977965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-4a4e882c-80b4-4ca9-b2f0-af2a0fd9c429 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:07.979275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4a4e882c-80b4-4ca9-b2f0-af2a0fd9c429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:08.060638 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 531/1061] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:02:08.075357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e5236-012b-46f0-9d20-dd0dd189e6eb None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:08.124398 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff56809-60eb-461a-a097-1aa3bbe6679a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:08.125375 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff56809-60eb-461a-a097-1aa3bbe6679a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:08.125870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff56809-60eb-461a-a097-1aa3bbe6679a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:08.126317 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff56809-60eb-461a-a097-1aa3bbe6679a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:08.136708 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff56809-60eb-461a-a097-1aa3bbe6679a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:08.138515 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff56809-60eb-461a-a097-1aa3bbe6679a None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-7ff56809-60eb-461a-a097-1aa3bbe6679a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:02:08.152917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:08.153967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:08.155048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:08.155048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:08.155048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:02:08.155048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:02:08.155048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:02:08.155048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:02:08.155048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:02:08.155048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:02:08.155048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:02:08.155048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:08.161746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 531/1062] 10.222.0.61 () {56 vars in 1093 bytes} [Wed Sep 7 13:02:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:08.164288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e882c-80b4-4ca9-b2f0-af2a0fd9c429 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:08.238229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:08.292944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e882c-80b4-4ca9-b2f0-af2a0fd9c429 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:08.339360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3b4a5-95fd-4720-ba1b-065b435bcc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:08.340288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff56809-60eb-461a-a097-1aa3bbe6679a None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:08.340501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3b4a5-95fd-4720-ba1b-065b435bcc6b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:08.341103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3b4a5-95fd-4720-ba1b-065b435bcc6b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:08.341527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3b4a5-95fd-4720-ba1b-065b435bcc6b None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:08.342437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3b4a5-95fd-4720-ba1b-065b435bcc6b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:08.342926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3b4a5-95fd-4720-ba1b-065b435bcc6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:08.344853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3b4a5-95fd-4720-ba1b-065b435bcc6b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:08.371483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 532/1063] 10.222.0.61 () {56 vars in 1133 bytes} [Wed Sep 7 13:02:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:08.383093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3b4a5-95fd-4720-ba1b-065b435bcc6b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:08.398357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:08.482915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff56809-60eb-461a-a097-1aa3bbe6679a None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:08.526146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c81339-d089-48a3-bd40-2d22816e60e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:08.527130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81339-d089-48a3-bd40-2d22816e60e0 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:08.527130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81339-d089-48a3-bd40-2d22816e60e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:08.527432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81339-d089-48a3-bd40-2d22816e60e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:08.537733 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3b4a5-95fd-4720-ba1b-065b435bcc6b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:08.540922 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c81339-d089-48a3-bd40-2d22816e60e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-16c81339-d089-48a3-bd40-2d22816e60e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:02:08.547296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:08.548435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:08.549285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:08.549285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:08.549285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:08.549285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:08.549285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:02:08.549285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:02:08.549285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:02:08.549285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:02:08.549285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:02:08.549285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:02:08.549285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:02:08.549285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:08.549285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 532/1064] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:02:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:02:08.555334 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c81339-d089-48a3-bd40-2d22816e60e0 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:08.569141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:08.602029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd80187-16ba-4004-8341-e7c1115a710e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:08.602978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd80187-16ba-4004-8341-e7c1115a710e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:08.603494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd80187-16ba-4004-8341-e7c1115a710e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:08.603979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd80187-16ba-4004-8341-e7c1115a710e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:08.624025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd80187-16ba-4004-8341-e7c1115a710e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:08.625770 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd80187-16ba-4004-8341-e7c1115a710e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-afd80187-16ba-4004-8341-e7c1115a710e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:02:08.646294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:08.647428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:02:08.648597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:02:08.648597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:08.648597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:02:08.648597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 533/1065] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:02:08.658639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd80187-16ba-4004-8341-e7c1115a710e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:08.694421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:08.704603 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c81339-d089-48a3-bd40-2d22816e60e0 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:08.749662 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb08df-edda-432f-8ec8-09aed5b56db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:08.750725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb08df-edda-432f-8ec8-09aed5b56db4 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:08.751265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb08df-edda-432f-8ec8-09aed5b56db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:08.751767 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb08df-edda-432f-8ec8-09aed5b56db4 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:08.753580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb08df-edda-432f-8ec8-09aed5b56db4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:08.754070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb08df-edda-432f-8ec8-09aed5b56db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:08.756772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb08df-edda-432f-8ec8-09aed5b56db4 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:08.773085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 533/1066] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:02:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:08.779385 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb08df-edda-432f-8ec8-09aed5b56db4 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:08.799513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd80187-16ba-4004-8341-e7c1115a710e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:08.865901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98316067-9fe8-4c2b-99ea-cda3ca689596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:08.867349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98316067-9fe8-4c2b-99ea-cda3ca689596 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:08.868018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98316067-9fe8-4c2b-99ea-cda3ca689596 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:08.868674 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98316067-9fe8-4c2b-99ea-cda3ca689596 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:08.870010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98316067-9fe8-4c2b-99ea-cda3ca689596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:08.870709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98316067-9fe8-4c2b-99ea-cda3ca689596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:08.873496 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98316067-9fe8-4c2b-99ea-cda3ca689596 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:08.895351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 534/1067] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:08.901179 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98316067-9fe8-4c2b-99ea-cda3ca689596 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:08.944898 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb08df-edda-432f-8ec8-09aed5b56db4 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:08.945113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:09.000998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050694b5-d45f-4ef3-9f2b-cbc5e92dc934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:09.002005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050694b5-d45f-4ef3-9f2b-cbc5e92dc934 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:09.002490 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050694b5-d45f-4ef3-9f2b-cbc5e92dc934 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:09.002963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050694b5-d45f-4ef3-9f2b-cbc5e92dc934 None admin] PATH_INFO: `/v3/users/dee0dca30734456da33c7b2ec0534ebe` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:09.005155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050694b5-d45f-4ef3-9f2b-cbc5e92dc934 None admin] RBAC: Authorizing `identity:get_user(user_id=dee0dca30734456da33c7b2ec0534ebe)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:09.007934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050694b5-d45f-4ef3-9f2b-cbc5e92dc934 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:09.015826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 534/1068] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:02:08 2022] GET /identity/v3/users/dee0dca30734456da33c7b2ec0534ebe => generated 285 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:09.017555 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050694b5-d45f-4ef3-9f2b-cbc5e92dc934 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:09.053849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98316067-9fe8-4c2b-99ea-cda3ca689596 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:09.063733 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:09.111637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a861a2-47f6-4dce-96c0-9f3a3d85db75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:09.113233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a861a2-47f6-4dce-96c0-9f3a3d85db75 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:09.113694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a861a2-47f6-4dce-96c0-9f3a3d85db75 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:09.114210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a861a2-47f6-4dce-96c0-9f3a3d85db75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:09.123934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a861a2-47f6-4dce-96c0-9f3a3d85db75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:09.125315 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a861a2-47f6-4dce-96c0-9f3a3d85db75 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-17a861a2-47f6-4dce-96c0-9f3a3d85db75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:02:09.139145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:09.146445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:09.147557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:02:09.147557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:02:09.147557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:02:09.147557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:02:09.147557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:02:09.147557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:02:09.147557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:02:09.147557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:02:09.147557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 535/1069] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:02:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:09.147557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a861a2-47f6-4dce-96c0-9f3a3d85db75 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:09.189572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050694b5-d45f-4ef3-9f2b-cbc5e92dc934 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:09.242810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2d806-b115-4d99-9719-f461b345a3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:09.243505 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2d806-b115-4d99-9719-f461b345a3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:09.244038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2d806-b115-4d99-9719-f461b345a3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:09.244318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2d806-b115-4d99-9719-f461b345a3a7 None admin] PATH_INFO: `/v3/projects/81e2949d0aea446998480e9f18835ee1` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:09.246044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2d806-b115-4d99-9719-f461b345a3a7 None admin] RBAC: Authorizing `identity:get_project(project_id=81e2949d0aea446998480e9f18835ee1)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:09.247699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2d806-b115-4d99-9719-f461b345a3a7 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:09.249727 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 535/1070] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:02:09 2022] GET /identity/v3/projects/81e2949d0aea446998480e9f18835ee1 => generated 311 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:09.256086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2d806-b115-4d99-9719-f461b345a3a7 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:09.296122 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:09.310807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a861a2-47f6-4dce-96c0-9f3a3d85db75 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:09.370534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8fd89-5d8d-4e48-bc3f-17db97be33ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:09.371734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8fd89-5d8d-4e48-bc3f-17db97be33ed None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:09.372181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8fd89-5d8d-4e48-bc3f-17db97be33ed None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:09.372636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8fd89-5d8d-4e48-bc3f-17db97be33ed None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:09.374110 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8fd89-5d8d-4e48-bc3f-17db97be33ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:09.374536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8fd89-5d8d-4e48-bc3f-17db97be33ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:09.376768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8fd89-5d8d-4e48-bc3f-17db97be33ed None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:09.398067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:09.398753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 536/1071] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:02:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:09.407793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8fd89-5d8d-4e48-bc3f-17db97be33ed None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:09.411205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2d806-b115-4d99-9719-f461b345a3a7 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:09.457103 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91740d05-dc90-4ed0-962d-6fa480f54a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:09.457849 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91740d05-dc90-4ed0-962d-6fa480f54a59 None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:09.458127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91740d05-dc90-4ed0-962d-6fa480f54a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:09.458401 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91740d05-dc90-4ed0-962d-6fa480f54a59 None admin] PATH_INFO: `/v3/projects/81e2949d0aea446998480e9f18835ee1/users/dee0dca30734456da33c7b2ec0534ebe/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:09.461150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91740d05-dc90-4ed0-962d-6fa480f54a59 None admin] RBAC: Authorizing `identity:create_grant(project_id=81e2949d0aea446998480e9f18835ee1, user_id=dee0dca30734456da33c7b2ec0534ebe, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:09.462995 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91740d05-dc90-4ed0-962d-6fa480f54a59 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:09.481094 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 536/1072] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:02:09 2022] PUT /identity/v3/projects/81e2949d0aea446998480e9f18835ee1/users/dee0dca30734456da33c7b2ec0534ebe/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:02:09.547364 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8fd89-5d8d-4e48-bc3f-17db97be33ed None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:09.595832 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3106f76-5acd-4d4c-ad54-d24005728642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:09.597475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3106f76-5acd-4d4c-ad54-d24005728642 None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:09.597972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3106f76-5acd-4d4c-ad54-d24005728642 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:09.598446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3106f76-5acd-4d4c-ad54-d24005728642 None admin] PATH_INFO: `/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/ba93e2d4aaed4c5595d5fd7363626bcb/roles/3fe314d875214a6485d3ff844b6cebf3` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:09.601282 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3106f76-5acd-4d4c-ad54-d24005728642 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user_id=ba93e2d4aaed4c5595d5fd7363626bcb, role_id=3fe314d875214a6485d3ff844b6cebf3)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:09.603403 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3106f76-5acd-4d4c-ad54-d24005728642 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:09.622584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 537/1073] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:02:09 2022] PUT /identity/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/ba93e2d4aaed4c5595d5fd7363626bcb/roles/3fe314d875214a6485d3ff844b6cebf3 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:02:09.738681 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60b1b8-753d-4a34-bc12-3313c36fa5c0 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:09.739252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60b1b8-753d-4a34-bc12-3313c36fa5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:09.739548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60b1b8-753d-4a34-bc12-3313c36fa5c0 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:09.742169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 537/1074] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:09.751622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237f704-769c-4819-a6bc-2b5340351c30 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:09.752067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237f704-769c-4819-a6bc-2b5340351c30 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:09.752628 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237f704-769c-4819-a6bc-2b5340351c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:09.780626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-6237f704-769c-4819-a6bc-2b5340351c30 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:09.782630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-6237f704-769c-4819-a6bc-2b5340351c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:09.788520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:09.927272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 538/1075] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:10.361071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60b1b8-753d-4a34-bc12-3313c36fa5c0 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:10.491939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60b1b8-753d-4a34-bc12-3313c36fa5c0 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:10.535967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2529f-29f2-47df-9057-0959937f6468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:10.536810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2529f-29f2-47df-9057-0959937f6468 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:10.537231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2529f-29f2-47df-9057-0959937f6468 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:10.537747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2529f-29f2-47df-9057-0959937f6468 None admin] PATH_INFO: `/v3/services/image` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:10.540161 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2529f-29f2-47df-9057-0959937f6468 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:10.543611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2529f-29f2-47df-9057-0959937f6468 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-dcc2529f-29f2-47df-9057-0959937f6468 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:10.561882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:10.568229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:10.570002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 538/1076] 10.222.0.61 () {56 vars in 1105 bytes} [Wed Sep 7 13:02:10 2022] GET /identity/v3/services/image => generated 86 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:10.572760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237f704-769c-4819-a6bc-2b5340351c30 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:10.697215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237f704-769c-4819-a6bc-2b5340351c30 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:10.743491 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c57227-60de-4c59-9f4c-09d5796a67ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:10.744277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c57227-60de-4c59-9f4c-09d5796a67ed None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:10.744649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c57227-60de-4c59-9f4c-09d5796a67ed None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:10.745056 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c57227-60de-4c59-9f4c-09d5796a67ed None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:10.745639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c57227-60de-4c59-9f4c-09d5796a67ed None admin] RBAC: Adding query filter params (name=image) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:10.745948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c57227-60de-4c59-9f4c-09d5796a67ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:10.747460 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c57227-60de-4c59-9f4c-09d5796a67ed None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:10.754169 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 539/1077] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:10.761032 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2529f-29f2-47df-9057-0959937f6468 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:10.790073 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:10.898408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2529f-29f2-47df-9057-0959937f6468 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:10.940306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018f004-c59c-41ef-a814-952aa6e7b4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:10.940932 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f004-c59c-41ef-a814-952aa6e7b4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:10.941209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f004-c59c-41ef-a814-952aa6e7b4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:10.941546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f004-c59c-41ef-a814-952aa6e7b4c0 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:10.942284 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018f004-c59c-41ef-a814-952aa6e7b4c0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:10.942589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018f004-c59c-41ef-a814-952aa6e7b4c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:10.944078 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018f004-c59c-41ef-a814-952aa6e7b4c0 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:10.952003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 539/1078] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:10.962268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c57227-60de-4c59-9f4c-09d5796a67ed None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:10.968887 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:11.137356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c57227-60de-4c59-9f4c-09d5796a67ed None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:11.179920 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a366893-76e5-4d6a-9727-c306e3c2222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:11.180551 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a366893-76e5-4d6a-9727-c306e3c2222b None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:11.180852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a366893-76e5-4d6a-9727-c306e3c2222b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:11.181129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a366893-76e5-4d6a-9727-c306e3c2222b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:11.181804 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a366893-76e5-4d6a-9727-c306e3c2222b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:11.183313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a366893-76e5-4d6a-9727-c306e3c2222b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:11.215249 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 540/1079] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:02:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:02:12.094216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407abed-148a-4855-9e31-57df32ebe5d6 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:12.095035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407abed-148a-4855-9e31-57df32ebe5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:12.095035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407abed-148a-4855-9e31-57df32ebe5d6 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:12.096549 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 540/1080] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:02:12.103286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7eef4-d961-46a0-98e6-05266d5eec96 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:12.103855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7eef4-d961-46a0-98e6-05266d5eec96 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:12.104175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7eef4-d961-46a0-98e6-05266d5eec96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:12.131049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-1cb7eef4-d961-46a0-98e6-05266d5eec96 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:12.132437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1cb7eef4-d961-46a0-98e6-05266d5eec96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:12.206105 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:12.242517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 541/1081] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:12.420508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef79ed0-78a4-4f09-8f39-a17eadad939c None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:12.420688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef79ed0-78a4-4f09-8f39-a17eadad939c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:12.420966 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef79ed0-78a4-4f09-8f39-a17eadad939c None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:12.423160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 541/1082] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:12.429063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc5f2c-9c8f-4534-8a94-7c727f5eb0b3 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:12.429432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc5f2c-9c8f-4534-8a94-7c727f5eb0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:12.429720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc5f2c-9c8f-4534-8a94-7c727f5eb0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:12.452485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-14dc5f2c-9c8f-4534-8a94-7c727f5eb0b3 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:12.454125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-14dc5f2c-9c8f-4534-8a94-7c727f5eb0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:12.539677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 542/1083] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:12.713252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef79ed0-78a4-4f09-8f39-a17eadad939c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:12.817700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:12.845186 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef79ed0-78a4-4f09-8f39-a17eadad939c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:12.889721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7a003-2644-4241-9050-e35083d41edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:12.890383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7a003-2644-4241-9050-e35083d41edc None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:12.890667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7a003-2644-4241-9050-e35083d41edc None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:12.890916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7a003-2644-4241-9050-e35083d41edc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:12.905240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7a003-2644-4241-9050-e35083d41edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-1de7a003-2644-4241-9050-e35083d41edc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:02:12.910975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:12.913035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:12.914821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:12.914821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:12.914821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:12.914821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:12.914821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:02:12.914821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:02:12.914821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:02:12.914821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:02:12.914821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:02:12.914821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:02:12.914821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:02:12.914821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:12.914821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 542/1084] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:02:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:12.922325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc5f2c-9c8f-4534-8a94-7c727f5eb0b3 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:13.047904 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7a003-2644-4241-9050-e35083d41edc None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:13.107088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc5f2c-9c8f-4534-8a94-7c727f5eb0b3 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:13.140536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:13.176363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8e920-c669-4b8a-98ff-7de9b5592076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:13.177563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8e920-c669-4b8a-98ff-7de9b5592076 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:13.178065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8e920-c669-4b8a-98ff-7de9b5592076 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:13.178541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8e920-c669-4b8a-98ff-7de9b5592076 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:13.179841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8e920-c669-4b8a-98ff-7de9b5592076 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:13.180379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8e920-c669-4b8a-98ff-7de9b5592076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:13.183603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8e920-c669-4b8a-98ff-7de9b5592076 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:13.208709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 543/1085] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:02:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:13.210351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8e920-c669-4b8a-98ff-7de9b5592076 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:13.223954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7a003-2644-4241-9050-e35083d41edc None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:13.287816 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a1ea3-bb9e-46ba-8a23-ee8c2ee9c619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:13.288859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a1ea3-bb9e-46ba-8a23-ee8c2ee9c619 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:13.289372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a1ea3-bb9e-46ba-8a23-ee8c2ee9c619 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:13.289808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a1ea3-bb9e-46ba-8a23-ee8c2ee9c619 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:13.308129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a1ea3-bb9e-46ba-8a23-ee8c2ee9c619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-7a3a1ea3-bb9e-46ba-8a23-ee8c2ee9c619 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:02:13.317344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:13.319537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:13.321279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:13.321279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:13.321279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:13.321279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:13.321279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:02:13.321279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:02:13.321279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:02:13.321279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:02:13.321279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:02:13.321279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:02:13.321279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:02:13.321279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:13.325548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 543/1086] 10.222.0.61 () {56 vars in 1120 bytes} [Wed Sep 7 13:02:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:13.326560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a1ea3-bb9e-46ba-8a23-ee8c2ee9c619 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:13.331933 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:13.388734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8e920-c669-4b8a-98ff-7de9b5592076 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:13.448402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b8ecc-cf9a-4e30-827e-e251e663b679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:13.449292 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b8ecc-cf9a-4e30-827e-e251e663b679 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:13.449659 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b8ecc-cf9a-4e30-827e-e251e663b679 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:13.450116 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b8ecc-cf9a-4e30-827e-e251e663b679 None admin] PATH_INFO: `/v3/users/dee0dca30734456da33c7b2ec0534ebe` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:13.451604 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b8ecc-cf9a-4e30-827e-e251e663b679 None admin] RBAC: Authorizing `identity:get_user(user_id=dee0dca30734456da33c7b2ec0534ebe)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:13.453416 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b8ecc-cf9a-4e30-827e-e251e663b679 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:13.461149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 544/1087] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:02:13 2022] GET /identity/v3/users/dee0dca30734456da33c7b2ec0534ebe => generated 285 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:13.462840 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b8ecc-cf9a-4e30-827e-e251e663b679 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:13.537588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:13.559710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a1ea3-bb9e-46ba-8a23-ee8c2ee9c619 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:13.614480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4b54d-7c80-446c-99f0-bd2c4919098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:13.615339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4b54d-7c80-446c-99f0-bd2c4919098c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:13.615811 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4b54d-7c80-446c-99f0-bd2c4919098c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:13.616238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4b54d-7c80-446c-99f0-bd2c4919098c None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:13.617093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4b54d-7c80-446c-99f0-bd2c4919098c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:13.617612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4b54d-7c80-446c-99f0-bd2c4919098c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:13.619552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4b54d-7c80-446c-99f0-bd2c4919098c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:13.637067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 544/1088] 10.222.0.61 () {56 vars in 1115 bytes} [Wed Sep 7 13:02:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:13.638747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4b54d-7c80-446c-99f0-bd2c4919098c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:13.652883 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b8ecc-cf9a-4e30-827e-e251e663b679 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:13.724606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c65dab-059d-404e-9dd8-4d81369e8ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:13.725716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c65dab-059d-404e-9dd8-4d81369e8ddc None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:13.726127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c65dab-059d-404e-9dd8-4d81369e8ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:13.726610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c65dab-059d-404e-9dd8-4d81369e8ddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:13.729269 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:13.789620 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4b54d-7c80-446c-99f0-bd2c4919098c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:13.857513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3b589-30af-47c9-b2ad-1cf1652cf1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:13.858426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3b589-30af-47c9-b2ad-1cf1652cf1da None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:13.858784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3b589-30af-47c9-b2ad-1cf1652cf1da None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:13.859028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3b589-30af-47c9-b2ad-1cf1652cf1da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:13.864250 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c65dab-059d-404e-9dd8-4d81369e8ddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:13.866995 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c65dab-059d-404e-9dd8-4d81369e8ddc None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:13.880224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3b589-30af-47c9-b2ad-1cf1652cf1da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:13.882435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3b589-30af-47c9-b2ad-1cf1652cf1da None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-fce3b589-30af-47c9-b2ad-1cf1652cf1da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:02:13.907954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:13.910333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:02:13.912183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:02:13.912183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:13.912183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:13.912183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 545/1089] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:13.919374 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3b589-30af-47c9-b2ad-1cf1652cf1da None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:13.957711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:14.070776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 545/1090] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:02:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:02:14.079849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c65dab-059d-404e-9dd8-4d81369e8ddc None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:14.091430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3b589-30af-47c9-b2ad-1cf1652cf1da None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:14.130889 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:14.170081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86208037-906f-48aa-9002-37a6824de978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:14.171094 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86208037-906f-48aa-9002-37a6824de978 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:14.171611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86208037-906f-48aa-9002-37a6824de978 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:14.172068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86208037-906f-48aa-9002-37a6824de978 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:14.173220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86208037-906f-48aa-9002-37a6824de978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:14.173782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86208037-906f-48aa-9002-37a6824de978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:14.176155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86208037-906f-48aa-9002-37a6824de978 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:14.201916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 546/1091] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:14.209839 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86208037-906f-48aa-9002-37a6824de978 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:14.250847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c65dab-059d-404e-9dd8-4d81369e8ddc None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:14.324286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd2593-e48e-4b48-bdde-bb6e57d8a646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:14.324957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd2593-e48e-4b48-bdde-bb6e57d8a646 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:14.325236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd2593-e48e-4b48-bdde-bb6e57d8a646 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:14.325524 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd2593-e48e-4b48-bdde-bb6e57d8a646 None admin] PATH_INFO: `/v3/projects/81e2949d0aea446998480e9f18835ee1` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:14.326812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd2593-e48e-4b48-bdde-bb6e57d8a646 None admin] RBAC: Authorizing `identity:get_project(project_id=81e2949d0aea446998480e9f18835ee1)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:14.328357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd2593-e48e-4b48-bdde-bb6e57d8a646 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:14.338970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 546/1092] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:02:14 2022] GET /identity/v3/projects/81e2949d0aea446998480e9f18835ee1 => generated 311 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:14.340613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd2593-e48e-4b48-bdde-bb6e57d8a646 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:14.367217 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:14.377772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86208037-906f-48aa-9002-37a6824de978 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:14.435181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e3fc5-1a2d-4e9e-9cb9-0fc3ef2a333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:14.435771 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e3fc5-1a2d-4e9e-9cb9-0fc3ef2a333b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:14.436053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e3fc5-1a2d-4e9e-9cb9-0fc3ef2a333b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:14.436309 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e3fc5-1a2d-4e9e-9cb9-0fc3ef2a333b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:14.447595 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e3fc5-1a2d-4e9e-9cb9-0fc3ef2a333b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:14.450137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e3fc5-1a2d-4e9e-9cb9-0fc3ef2a333b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-463e3fc5-1a2d-4e9e-9cb9-0fc3ef2a333b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:02:14.471690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:14.473485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:14.474622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:14.474622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:14.474622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:02:14.474622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:02:14.474622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:02:14.474622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:02:14.474622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:02:14.474622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:02:14.474622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:02:14.474622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:14.474622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 547/1093] 10.222.0.61 () {56 vars in 1093 bytes} [Wed Sep 7 13:02:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:14.476085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cface4-bcd7-4079-837b-c3b9e6d4835b None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:14.476366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cface4-bcd7-4079-837b-c3b9e6d4835b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:14.476796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cface4-bcd7-4079-837b-c3b9e6d4835b None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:14.479823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 548/1094] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:14.484445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cface4-bcd7-4079-837b-c3b9e6d4835b None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:14.509448 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd2593-e48e-4b48-bdde-bb6e57d8a646 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:14.549772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:14.570417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e86e77-f4b2-45ea-a43a-76eb8fe4a3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:14.571247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e86e77-f4b2-45ea-a43a-76eb8fe4a3af None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:14.571617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e86e77-f4b2-45ea-a43a-76eb8fe4a3af None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:14.571973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e86e77-f4b2-45ea-a43a-76eb8fe4a3af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:14.573000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e86e77-f4b2-45ea-a43a-76eb8fe4a3af None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=81e2949d0aea446998480e9f18835ee1, user.id=dee0dca30734456da33c7b2ec0534ebe) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:14.573400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e86e77-f4b2-45ea-a43a-76eb8fe4a3af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:14.575813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e86e77-f4b2-45ea-a43a-76eb8fe4a3af None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:14.586770 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 547/1095] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:02:14 2022] GET /identity/v3/role_assignments?user.id=dee0dca30734456da33c7b2ec0534ebe&scope.project.id=81e2949d0aea446998480e9f18835ee1&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:14.588366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcc2b0-a6e3-46b4-a525-c1205e2de8bf None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:14.588651 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcc2b0-a6e3-46b4-a525-c1205e2de8bf None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:14.588924 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcc2b0-a6e3-46b4-a525-c1205e2de8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:14.616089 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-e8fcc2b0-a6e3-46b4-a525-c1205e2de8bf None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:14.618115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e8fcc2b0-a6e3-46b4-a525-c1205e2de8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:14.671489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cface4-bcd7-4079-837b-c3b9e6d4835b None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:14.719164 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217f1bc-2322-46a9-9958-3e1f209bdaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:14.719787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217f1bc-2322-46a9-9958-3e1f209bdaac None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:14.720061 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217f1bc-2322-46a9-9958-3e1f209bdaac None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:14.720395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217f1bc-2322-46a9-9958-3e1f209bdaac None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:14.722025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217f1bc-2322-46a9-9958-3e1f209bdaac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:14.722355 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217f1bc-2322-46a9-9958-3e1f209bdaac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:14.723937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217f1bc-2322-46a9-9958-3e1f209bdaac None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:14.755145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 549/1096] 10.222.0.61 () {56 vars in 1133 bytes} [Wed Sep 7 13:02:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:14.758995 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217f1bc-2322-46a9-9958-3e1f209bdaac None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:14.775176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 548/1097] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:14.930760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:14.941656 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217f1bc-2322-46a9-9958-3e1f209bdaac None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:14.986119 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812c96e-1aa9-4a4e-9687-7a508acc0d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:14.986862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812c96e-1aa9-4a4e-9687-7a508acc0d3a None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:14.987306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812c96e-1aa9-4a4e-9687-7a508acc0d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:14.987707 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812c96e-1aa9-4a4e-9687-7a508acc0d3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:15.086406 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812c96e-1aa9-4a4e-9687-7a508acc0d3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:15.089214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812c96e-1aa9-4a4e-9687-7a508acc0d3a None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:15.167529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcc2b0-a6e3-46b4-a525-c1205e2de8bf None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:15.190940 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:15.245269 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:15.260491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 550/1098] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:02:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:02:15.266620 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812c96e-1aa9-4a4e-9687-7a508acc0d3a None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:15.350932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcc2b0-a6e3-46b4-a525-c1205e2de8bf None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:15.406216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b891a9-ce21-4359-9c40-4edff2b875b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:15.407147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b891a9-ce21-4359-9c40-4edff2b875b2 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:15.407617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b891a9-ce21-4359-9c40-4edff2b875b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:15.408045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b891a9-ce21-4359-9c40-4edff2b875b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:15.416373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812c96e-1aa9-4a4e-9687-7a508acc0d3a None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:15.460397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdec9d9-432e-4d08-8325-e7f2d7d0e770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:15.461122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdec9d9-432e-4d08-8325-e7f2d7d0e770 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:15.461401 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdec9d9-432e-4d08-8325-e7f2d7d0e770 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:15.461664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdec9d9-432e-4d08-8325-e7f2d7d0e770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:15.476294 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdec9d9-432e-4d08-8325-e7f2d7d0e770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:15.477856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdec9d9-432e-4d08-8325-e7f2d7d0e770 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-bfdec9d9-432e-4d08-8325-e7f2d7d0e770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:02:15.494781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:15.495916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:02:15.496836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:02:15.496836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:15.496836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:15.496836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 551/1099] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:15.505712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdec9d9-432e-4d08-8325-e7f2d7d0e770 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:15.509709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b891a9-ce21-4359-9c40-4edff2b875b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:15.511561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b891a9-ce21-4359-9c40-4edff2b875b2 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:15.523784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:15.571783 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:15.645823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdec9d9-432e-4d08-8325-e7f2d7d0e770 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:15.674406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 549/1100] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:02:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 13:02:15.680357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b891a9-ce21-4359-9c40-4edff2b875b2 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:15.688223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf637b4-6754-4d53-b3e8-5727d80e2e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:15.688842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf637b4-6754-4d53-b3e8-5727d80e2e06 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:15.689105 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf637b4-6754-4d53-b3e8-5727d80e2e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:15.690599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf637b4-6754-4d53-b3e8-5727d80e2e06 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:15.690762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf637b4-6754-4d53-b3e8-5727d80e2e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:15.691020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf637b4-6754-4d53-b3e8-5727d80e2e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:15.692423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf637b4-6754-4d53-b3e8-5727d80e2e06 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:15.713771 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 552/1101] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:15.720044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf637b4-6754-4d53-b3e8-5727d80e2e06 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:15.844899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b891a9-ce21-4359-9c40-4edff2b875b2 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:15.863216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:15.894970 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf637b4-6754-4d53-b3e8-5727d80e2e06 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:15.898791 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c260ca-2add-4756-b490-a522c4608fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:15.899695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c260ca-2add-4756-b490-a522c4608fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:15.900109 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c260ca-2add-4756-b490-a522c4608fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:15.900524 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c260ca-2add-4756-b490-a522c4608fc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:15.906155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:15.917232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c260ca-2add-4756-b490-a522c4608fc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:15.919347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c260ca-2add-4756-b490-a522c4608fc3 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-83c260ca-2add-4756-b490-a522c4608fc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:02:15.939063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:15.941703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:02:15.945446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:02:15.945446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:02:15.945446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:02:15.945446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 550/1102] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:02:15.946693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a34eaa2-b560-4909-8adf-a1e053b4de82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:15.946693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34eaa2-b560-4909-8adf-a1e053b4de82 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:15.946693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34eaa2-b560-4909-8adf-a1e053b4de82 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:15.947409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34eaa2-b560-4909-8adf-a1e053b4de82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:15.948344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c260ca-2add-4756-b490-a522c4608fc3 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:15.956898 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34eaa2-b560-4909-8adf-a1e053b4de82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:15.958451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34eaa2-b560-4909-8adf-a1e053b4de82 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-7a34eaa2-b560-4909-8adf-a1e053b4de82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:02:15.971530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:15.973865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:15.975876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:02:15.975876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:02:15.975876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:02:15.975876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:02:15.975876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:02:15.975876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:02:15.975876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:02:15.975876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:15.975876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 553/1103] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:02:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:02:15.980281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a34eaa2-b560-4909-8adf-a1e053b4de82 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:16.121961 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c260ca-2add-4756-b490-a522c4608fc3 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:16.148403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a34eaa2-b560-4909-8adf-a1e053b4de82 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:16.165532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953be5d1-c31f-40b2-9fe8-61968ae5c95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:16.166318 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953be5d1-c31f-40b2-9fe8-61968ae5c95b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:16.166633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953be5d1-c31f-40b2-9fe8-61968ae5c95b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:16.166907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953be5d1-c31f-40b2-9fe8-61968ae5c95b None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:16.167553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953be5d1-c31f-40b2-9fe8-61968ae5c95b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:16.167849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953be5d1-c31f-40b2-9fe8-61968ae5c95b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:16.169330 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953be5d1-c31f-40b2-9fe8-61968ae5c95b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:16.195141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 551/1104] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:16.215609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271df3a-0423-4fb1-9433-399b87b2844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:16.216707 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271df3a-0423-4fb1-9433-399b87b2844e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:16.217207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271df3a-0423-4fb1-9433-399b87b2844e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:16.217654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271df3a-0423-4fb1-9433-399b87b2844e None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:16.218960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271df3a-0423-4fb1-9433-399b87b2844e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:16.219527 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271df3a-0423-4fb1-9433-399b87b2844e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:16.222849 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271df3a-0423-4fb1-9433-399b87b2844e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:16.248047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 554/1105] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:02:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:16.254861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953be5d1-c31f-40b2-9fe8-61968ae5c95b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:16.286620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:16.398061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953be5d1-c31f-40b2-9fe8-61968ae5c95b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:16.440589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4281d9-7e06-4aeb-822d-5b066f661e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:16.441218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4281d9-7e06-4aeb-822d-5b066f661e70 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:16.441494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4281d9-7e06-4aeb-822d-5b066f661e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:16.441766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4281d9-7e06-4aeb-822d-5b066f661e70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:16.442517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4281d9-7e06-4aeb-822d-5b066f661e70 None admin] RBAC: Adding query filter params (role.id=3fe314d875214a6485d3ff844b6cebf3, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=ba93e2d4aaed4c5595d5fd7363626bcb) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:16.442820 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4281d9-7e06-4aeb-822d-5b066f661e70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:16.444327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4281d9-7e06-4aeb-822d-5b066f661e70 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:16.451334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 552/1106] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:02:16 2022] GET /identity/v3/role_assignments?user.id=ba93e2d4aaed4c5595d5fd7363626bcb&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=3fe314d875214a6485d3ff844b6cebf3 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:17.839051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24272dd-6592-4acc-8174-86a4e939f1b2 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:17.839715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24272dd-6592-4acc-8174-86a4e939f1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:17.839845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24272dd-6592-4acc-8174-86a4e939f1b2 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:17.841119 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 555/1107] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:17.847188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527acfd2-f57a-40fe-9e56-bbd14e0d8849 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:17.847782 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527acfd2-f57a-40fe-9e56-bbd14e0d8849 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:17.847782 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527acfd2-f57a-40fe-9e56-bbd14e0d8849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:17.867321 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-527acfd2-f57a-40fe-9e56-bbd14e0d8849 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:17.868558 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-527acfd2-f57a-40fe-9e56-bbd14e0d8849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:17.945459 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 553/1108] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:18.219648 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24272dd-6592-4acc-8174-86a4e939f1b2 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:18.252756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:18.339485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24272dd-6592-4acc-8174-86a4e939f1b2 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:18.380086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3e133-414d-4d8f-bc30-cd9858b6dc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:18.380737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3e133-414d-4d8f-bc30-cd9858b6dc8e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:18.381016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3e133-414d-4d8f-bc30-cd9858b6dc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:18.381283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3e133-414d-4d8f-bc30-cd9858b6dc8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:18.382514 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3e133-414d-4d8f-bc30-cd9858b6dc8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:18.384028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3e133-414d-4d8f-bc30-cd9858b6dc8e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:18.385686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 556/1109] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:18.392184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527acfd2-f57a-40fe-9e56-bbd14e0d8849 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:18.409839 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:18.511732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527acfd2-f57a-40fe-9e56-bbd14e0d8849 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:18.551860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398591b5-d7af-4c1a-9a31-c1c5dbb9d1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:18.552482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398591b5-d7af-4c1a-9a31-c1c5dbb9d1e5 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:18.552761 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398591b5-d7af-4c1a-9a31-c1c5dbb9d1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:18.553033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398591b5-d7af-4c1a-9a31-c1c5dbb9d1e5 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:18.553714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398591b5-d7af-4c1a-9a31-c1c5dbb9d1e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:18.555220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398591b5-d7af-4c1a-9a31-c1c5dbb9d1e5 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:18.580091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 554/1110] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 13:02:18 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:02:21.025276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe398466-8376-4670-bff3-3c49b5e66772 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:21.025894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe398466-8376-4670-bff3-3c49b5e66772 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:21.025894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe398466-8376-4670-bff3-3c49b5e66772 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:21.027422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 557/1111] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:02:21.034560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750061a-2007-4db9-9e2e-44493d40fc55 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:21.035300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750061a-2007-4db9-9e2e-44493d40fc55 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:21.035300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750061a-2007-4db9-9e2e-44493d40fc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:21.056323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-e750061a-2007-4db9-9e2e-44493d40fc55 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:21.057549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e750061a-2007-4db9-9e2e-44493d40fc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:21.133507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:21.140314 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 555/1112] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:21.427391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe398466-8376-4670-bff3-3c49b5e66772 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:21.549418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:21.558959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe398466-8376-4670-bff3-3c49b5e66772 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:21.598839 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa39d6-a152-4688-9085-4a7f97685f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:21.599526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa39d6-a152-4688-9085-4a7f97685f50 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:21.599801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa39d6-a152-4688-9085-4a7f97685f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:21.600072 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa39d6-a152-4688-9085-4a7f97685f50 None admin] PATH_INFO: `/v3/roles/649721c72e5d40deabd3512c529ffdf4` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:21.601562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa39d6-a152-4688-9085-4a7f97685f50 None admin] RBAC: Authorizing `identity:get_role(role_id=649721c72e5d40deabd3512c529ffdf4)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:21.603067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa39d6-a152-4688-9085-4a7f97685f50 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:21.604487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 558/1113] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:02:21 2022] GET /identity/v3/roles/649721c72e5d40deabd3512c529ffdf4 => generated 223 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:21.610692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750061a-2007-4db9-9e2e-44493d40fc55 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:21.732994 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750061a-2007-4db9-9e2e-44493d40fc55 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:21.773385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d2fe7-a4c7-43f4-8235-c68cfc15800f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:21.774042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d2fe7-a4c7-43f4-8235-c68cfc15800f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:21.774328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d2fe7-a4c7-43f4-8235-c68cfc15800f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:21.774600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d2fe7-a4c7-43f4-8235-c68cfc15800f None admin] PATH_INFO: `/v3/users/128a42a093594474a5760640b0b45a35` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:21.791139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d2fe7-a4c7-43f4-8235-c68cfc15800f None admin] RBAC: Authorizing `identity:get_user(user_id=128a42a093594474a5760640b0b45a35)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:21.792613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d2fe7-a4c7-43f4-8235-c68cfc15800f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:21.794348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 556/1114] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:02:21 2022] GET /identity/v3/users/128a42a093594474a5760640b0b45a35 => generated 286 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:21.804854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa39d6-a152-4688-9085-4a7f97685f50 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:21.943174 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa39d6-a152-4688-9085-4a7f97685f50 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:21.981700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60473f0b-7604-4d16-8777-f729f34caea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:21.982458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60473f0b-7604-4d16-8777-f729f34caea4 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:21.982824 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60473f0b-7604-4d16-8777-f729f34caea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:21.983294 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60473f0b-7604-4d16-8777-f729f34caea4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:22.018975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:22.065480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60473f0b-7604-4d16-8777-f729f34caea4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:22.066963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60473f0b-7604-4d16-8777-f729f34caea4 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:22.188148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 559/1115] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:02:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:02:22.194981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d2fe7-a4c7-43f4-8235-c68cfc15800f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:22.263897 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:22.310908 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d2fe7-a4c7-43f4-8235-c68cfc15800f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:22.352349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefecd37-b8a8-4efd-be88-f2a148949f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:22.352999 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefecd37-b8a8-4efd-be88-f2a148949f55 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:22.353480 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefecd37-b8a8-4efd-be88-f2a148949f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:22.353754 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefecd37-b8a8-4efd-be88-f2a148949f55 None admin] PATH_INFO: `/v3/projects/81e2949d0aea446998480e9f18835ee1` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:22.354965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefecd37-b8a8-4efd-be88-f2a148949f55 None admin] RBAC: Authorizing `identity:get_project(project_id=81e2949d0aea446998480e9f18835ee1)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:22.356523 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefecd37-b8a8-4efd-be88-f2a148949f55 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:22.358515 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 557/1116] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:02:22 2022] GET /identity/v3/projects/81e2949d0aea446998480e9f18835ee1 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:22.369806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60473f0b-7604-4d16-8777-f729f34caea4 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:22.484968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:22.535800 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60473f0b-7604-4d16-8777-f729f34caea4 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:22.573935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20762fe-7b48-4365-8ba1-4470bbeb12c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:22.574919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20762fe-7b48-4365-8ba1-4470bbeb12c5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:22.575480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20762fe-7b48-4365-8ba1-4470bbeb12c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:22.575954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20762fe-7b48-4365-8ba1-4470bbeb12c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:22.577305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20762fe-7b48-4365-8ba1-4470bbeb12c5 None admin] RBAC: Adding query filter params (role.id=649721c72e5d40deabd3512c529ffdf4, scope.project.id=81e2949d0aea446998480e9f18835ee1, user.id=128a42a093594474a5760640b0b45a35) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:22.577859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20762fe-7b48-4365-8ba1-4470bbeb12c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:22.580801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20762fe-7b48-4365-8ba1-4470bbeb12c5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:22.591217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 560/1117] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:02:22 2022] GET /identity/v3/role_assignments?user.id=128a42a093594474a5760640b0b45a35&scope.project.id=81e2949d0aea446998480e9f18835ee1&role.id=649721c72e5d40deabd3512c529ffdf4 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:25.457969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abde42-5437-4648-bdf1-d24add38b0e5 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:25.458501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abde42-5437-4648-bdf1-d24add38b0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:25.458871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abde42-5437-4648-bdf1-d24add38b0e5 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:25.461587 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 558/1118] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:25.468099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5c689-5c61-437a-a6da-67c59f432698 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:25.468716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5c689-5c61-437a-a6da-67c59f432698 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:25.468716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5c689-5c61-437a-a6da-67c59f432698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:25.489002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8aa5c689-5c61-437a-a6da-67c59f432698 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:25.490235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8aa5c689-5c61-437a-a6da-67c59f432698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:25.570985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 561/1119] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:25.854679 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abde42-5437-4648-bdf1-d24add38b0e5 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:25.973149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abde42-5437-4648-bdf1-d24add38b0e5 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:26.004133 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:26.014296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52a980-d3b6-43e0-aa62-735436fd5d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:26.014950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52a980-d3b6-43e0-aa62-735436fd5d60 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:26.015274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52a980-d3b6-43e0-aa62-735436fd5d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:26.015554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52a980-d3b6-43e0-aa62-735436fd5d60 None admin] PATH_INFO: `/v3/roles/649721c72e5d40deabd3512c529ffdf4` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:26.017639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52a980-d3b6-43e0-aa62-735436fd5d60 None admin] RBAC: Authorizing `identity:get_role(role_id=649721c72e5d40deabd3512c529ffdf4)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:26.019138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52a980-d3b6-43e0-aa62-735436fd5d60 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:26.020733 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 559/1120] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:02:25 2022] GET /identity/v3/roles/649721c72e5d40deabd3512c529ffdf4 => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:26.029849 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5c689-5c61-437a-a6da-67c59f432698 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:26.100300 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:26.152799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5c689-5c61-437a-a6da-67c59f432698 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:26.194001 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c14e83-c6c0-4d92-9c48-b63d42046727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:26.194713 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c14e83-c6c0-4d92-9c48-b63d42046727 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:26.195031 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c14e83-c6c0-4d92-9c48-b63d42046727 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:26.195624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c14e83-c6c0-4d92-9c48-b63d42046727 None admin] PATH_INFO: `/v3/users/128a42a093594474a5760640b0b45a35` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:26.196972 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c14e83-c6c0-4d92-9c48-b63d42046727 None admin] RBAC: Authorizing `identity:get_user(user_id=128a42a093594474a5760640b0b45a35)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:26.198506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c14e83-c6c0-4d92-9c48-b63d42046727 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:26.200575 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 562/1121] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:02:26 2022] GET /identity/v3/users/128a42a093594474a5760640b0b45a35 => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:26.208014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52a980-d3b6-43e0-aa62-735436fd5d60 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:26.322291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52a980-d3b6-43e0-aa62-735436fd5d60 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:26.361155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643a13a-5317-48d3-94f4-82b0e89d91fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:26.361812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643a13a-5317-48d3-94f4-82b0e89d91fd None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:26.362091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643a13a-5317-48d3-94f4-82b0e89d91fd None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:26.362360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643a13a-5317-48d3-94f4-82b0e89d91fd None admin] PATH_INFO: `/v3/projects/81e2949d0aea446998480e9f18835ee1` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:26.363634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643a13a-5317-48d3-94f4-82b0e89d91fd None admin] RBAC: Authorizing `identity:get_project(project_id=81e2949d0aea446998480e9f18835ee1)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:26.365101 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643a13a-5317-48d3-94f4-82b0e89d91fd None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:26.366833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 560/1122] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:02:26 2022] GET /identity/v3/projects/81e2949d0aea446998480e9f18835ee1 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:26.373403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c14e83-c6c0-4d92-9c48-b63d42046727 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:26.491224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c14e83-c6c0-4d92-9c48-b63d42046727 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:26.519562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:26.532309 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b86c7-3fe5-499b-b196-eb5865a23aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:26.533557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b86c7-3fe5-499b-b196-eb5865a23aa3 None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:26.533835 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b86c7-3fe5-499b-b196-eb5865a23aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:26.534109 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b86c7-3fe5-499b-b196-eb5865a23aa3 None admin] PATH_INFO: `/v3/projects/81e2949d0aea446998480e9f18835ee1/users/128a42a093594474a5760640b0b45a35/roles/649721c72e5d40deabd3512c529ffdf4` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:26.536700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b86c7-3fe5-499b-b196-eb5865a23aa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=81e2949d0aea446998480e9f18835ee1, user_id=128a42a093594474a5760640b0b45a35, role_id=649721c72e5d40deabd3512c529ffdf4)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:26.538186 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b86c7-3fe5-499b-b196-eb5865a23aa3 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:26.553595 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 563/1123] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:02:26 2022] PUT /identity/v3/projects/81e2949d0aea446998480e9f18835ee1/users/128a42a093594474a5760640b0b45a35/roles/649721c72e5d40deabd3512c529ffdf4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:02:28.968280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697bff7-5282-4ebb-b574-9ddfe7e3bbb8 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:28.968894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697bff7-5282-4ebb-b574-9ddfe7e3bbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:28.968894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697bff7-5282-4ebb-b574-9ddfe7e3bbb8 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:28.970934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 561/1124] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:02:28.977357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f36596-5255-4007-832b-7da1d098ae3d None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:28.978039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f36596-5255-4007-832b-7da1d098ae3d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:28.978039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f36596-5255-4007-832b-7da1d098ae3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:28.998231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-e0f36596-5255-4007-832b-7da1d098ae3d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:29.000298 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e0f36596-5255-4007-832b-7da1d098ae3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:29.089269 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 564/1125] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:29.392783 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697bff7-5282-4ebb-b574-9ddfe7e3bbb8 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:29.468218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:29.521632 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697bff7-5282-4ebb-b574-9ddfe7e3bbb8 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:29.563711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0f872-a651-45e6-a9c1-e5c572d7b330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:29.564379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0f872-a651-45e6-a9c1-e5c572d7b330 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:29.564667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0f872-a651-45e6-a9c1-e5c572d7b330 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:29.564941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0f872-a651-45e6-a9c1-e5c572d7b330 None admin] PATH_INFO: `/v3/roles/649721c72e5d40deabd3512c529ffdf4` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:29.566492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0f872-a651-45e6-a9c1-e5c572d7b330 None admin] RBAC: Authorizing `identity:get_role(role_id=649721c72e5d40deabd3512c529ffdf4)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:29.567945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0f872-a651-45e6-a9c1-e5c572d7b330 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:29.569381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 562/1126] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:02:29 2022] GET /identity/v3/roles/649721c72e5d40deabd3512c529ffdf4 => generated 223 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:29.580012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f36596-5255-4007-832b-7da1d098ae3d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:29.769233 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f36596-5255-4007-832b-7da1d098ae3d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:29.799883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:29.826843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aba13f-32e3-44b5-921d-8519d8d34fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:29.827570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aba13f-32e3-44b5-921d-8519d8d34fcf None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:29.827855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aba13f-32e3-44b5-921d-8519d8d34fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:29.828127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aba13f-32e3-44b5-921d-8519d8d34fcf None admin] PATH_INFO: `/v3/users/128a42a093594474a5760640b0b45a35` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:29.829468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aba13f-32e3-44b5-921d-8519d8d34fcf None admin] RBAC: Authorizing `identity:get_user(user_id=128a42a093594474a5760640b0b45a35)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:29.830962 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aba13f-32e3-44b5-921d-8519d8d34fcf None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:29.833106 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 565/1127] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:02:29 2022] GET /identity/v3/users/128a42a093594474a5760640b0b45a35 => generated 286 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:29.842342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0f872-a651-45e6-a9c1-e5c572d7b330 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:29.962352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0f872-a651-45e6-a9c1-e5c572d7b330 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:30.004444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:30.014889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005bb3c-aaf7-4f46-8eea-0f0ea7377782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:30.015943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005bb3c-aaf7-4f46-8eea-0f0ea7377782 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:30.016427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005bb3c-aaf7-4f46-8eea-0f0ea7377782 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:30.017547 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005bb3c-aaf7-4f46-8eea-0f0ea7377782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:30.120290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005bb3c-aaf7-4f46-8eea-0f0ea7377782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:30.121777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005bb3c-aaf7-4f46-8eea-0f0ea7377782 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:30.241527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 563/1128] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:02:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 13:02:30.248413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aba13f-32e3-44b5-921d-8519d8d34fcf None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:30.368315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aba13f-32e3-44b5-921d-8519d8d34fcf None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:30.408430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79760be7-e737-45e8-b3b9-b4dbd7d91181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:30.409138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79760be7-e737-45e8-b3b9-b4dbd7d91181 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:30.409459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79760be7-e737-45e8-b3b9-b4dbd7d91181 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:30.409770 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79760be7-e737-45e8-b3b9-b4dbd7d91181 None admin] PATH_INFO: `/v3/projects/81e2949d0aea446998480e9f18835ee1` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:30.411035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79760be7-e737-45e8-b3b9-b4dbd7d91181 None admin] RBAC: Authorizing `identity:get_project(project_id=81e2949d0aea446998480e9f18835ee1)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:30.412763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79760be7-e737-45e8-b3b9-b4dbd7d91181 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:30.414566 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 566/1129] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:02:30 2022] GET /identity/v3/projects/81e2949d0aea446998480e9f18835ee1 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:30.421393 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005bb3c-aaf7-4f46-8eea-0f0ea7377782 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:30.456056 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:30.537984 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005bb3c-aaf7-4f46-8eea-0f0ea7377782 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:30.580254 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35272f65-ac5c-454a-a950-9e464877fb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:30.580885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35272f65-ac5c-454a-a950-9e464877fb13 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:30.581170 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35272f65-ac5c-454a-a950-9e464877fb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:30.581447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35272f65-ac5c-454a-a950-9e464877fb13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:30.582203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35272f65-ac5c-454a-a950-9e464877fb13 None admin] RBAC: Adding query filter params (role.id=649721c72e5d40deabd3512c529ffdf4, scope.project.id=81e2949d0aea446998480e9f18835ee1, user.id=128a42a093594474a5760640b0b45a35) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:30.584509 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35272f65-ac5c-454a-a950-9e464877fb13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:30.586030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35272f65-ac5c-454a-a950-9e464877fb13 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:30.593483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 564/1130] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:02:30 2022] GET /identity/v3/role_assignments?user.id=128a42a093594474a5760640b0b45a35&scope.project.id=81e2949d0aea446998480e9f18835ee1&role.id=649721c72e5d40deabd3512c529ffdf4 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:33.059449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e0d7b-71e5-4182-88ac-81e82bc6d4bc None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:33.060336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e0d7b-71e5-4182-88ac-81e82bc6d4bc None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:33.060336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e0d7b-71e5-4182-88ac-81e82bc6d4bc None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:33.061380 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 567/1131] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:33.068025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca0846-8907-4a11-8101-d57f757f3cf4 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:33.068781 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca0846-8907-4a11-8101-d57f757f3cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:33.068781 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca0846-8907-4a11-8101-d57f757f3cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:33.089767 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-faca0846-8907-4a11-8101-d57f757f3cf4 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:33.091013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-faca0846-8907-4a11-8101-d57f757f3cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:33.197217 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 565/1132] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:33.484585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e0d7b-71e5-4182-88ac-81e82bc6d4bc None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:33.546084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:33.606227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e0d7b-71e5-4182-88ac-81e82bc6d4bc None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:33.647353 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1086c-5777-46de-9867-f65b5cb1e11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:33.648001 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1086c-5777-46de-9867-f65b5cb1e11b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:33.648277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1086c-5777-46de-9867-f65b5cb1e11b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:33.648546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1086c-5777-46de-9867-f65b5cb1e11b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:33.649762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1086c-5777-46de-9867-f65b5cb1e11b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:33.651655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1086c-5777-46de-9867-f65b5cb1e11b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:33.653488 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 568/1133] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:33.660324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca0846-8907-4a11-8101-d57f757f3cf4 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:33.689995 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:33.779862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca0846-8907-4a11-8101-d57f757f3cf4 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:33.818307 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4ad28-ea13-45cd-b6f1-cea6e2789778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:33.818942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4ad28-ea13-45cd-b6f1-cea6e2789778 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:33.819350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4ad28-ea13-45cd-b6f1-cea6e2789778 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:33.819681 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4ad28-ea13-45cd-b6f1-cea6e2789778 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:33.820693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4ad28-ea13-45cd-b6f1-cea6e2789778 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:33.822444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4ad28-ea13-45cd-b6f1-cea6e2789778 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:33.856872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 566/1134] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 13:02:33 2022] POST /identity/v3/projects => generated 311 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:02:36.204238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53293464-e32b-4281-9cc8-107268a042fa None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:36.204851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53293464-e32b-4281-9cc8-107268a042fa None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:36.204851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53293464-e32b-4281-9cc8-107268a042fa None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:36.206221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 569/1135] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:36.212473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bba4d-d2bb-4416-b376-ecfb1938d468 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:36.213047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bba4d-d2bb-4416-b376-ecfb1938d468 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:36.213047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bba4d-d2bb-4416-b376-ecfb1938d468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:36.233580 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-699bba4d-d2bb-4416-b376-ecfb1938d468 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:36.234898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-699bba4d-d2bb-4416-b376-ecfb1938d468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:36.315335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 567/1136] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:36.593536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53293464-e32b-4281-9cc8-107268a042fa None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:36.712862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53293464-e32b-4281-9cc8-107268a042fa None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:36.733446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:36.754142 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b4512-1434-4e5d-8db8-17e3128cf257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:36.754857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b4512-1434-4e5d-8db8-17e3128cf257 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:36.755237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b4512-1434-4e5d-8db8-17e3128cf257 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:36.755562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b4512-1434-4e5d-8db8-17e3128cf257 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:36.756802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b4512-1434-4e5d-8db8-17e3128cf257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:36.758324 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b4512-1434-4e5d-8db8-17e3128cf257 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:36.760258 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 570/1137] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:02:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:36.767069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bba4d-d2bb-4416-b376-ecfb1938d468 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:36.776445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:36.885349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bba4d-d2bb-4416-b376-ecfb1938d468 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:36.923968 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee09c6-5501-49c2-a9dc-d81e82388698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:36.924599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee09c6-5501-49c2-a9dc-d81e82388698 None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:36.924883 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee09c6-5501-49c2-a9dc-d81e82388698 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:36.925158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee09c6-5501-49c2-a9dc-d81e82388698 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:36.925842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee09c6-5501-49c2-a9dc-d81e82388698 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:36.927336 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee09c6-5501-49c2-a9dc-d81e82388698 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:36.953008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 568/1138] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 13:02:36 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:02:39.326203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882aad9-e21d-44b7-8c57-dab0cdf2df9d None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:39.326733 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882aad9-e21d-44b7-8c57-dab0cdf2df9d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:39.326733 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882aad9-e21d-44b7-8c57-dab0cdf2df9d None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:39.328349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 571/1139] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:39.335324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93678edb-1922-4cac-a9ad-1381491aabfa None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:39.336132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93678edb-1922-4cac-a9ad-1381491aabfa None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:39.336132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93678edb-1922-4cac-a9ad-1381491aabfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:39.358358 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-93678edb-1922-4cac-a9ad-1381491aabfa None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:39.359643 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-93678edb-1922-4cac-a9ad-1381491aabfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:39.430376 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:39.444061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 569/1140] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:39.723200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882aad9-e21d-44b7-8c57-dab0cdf2df9d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:39.843864 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882aad9-e21d-44b7-8c57-dab0cdf2df9d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:39.886621 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97339e3c-97f8-42ba-8072-95debac20cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:39.887308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97339e3c-97f8-42ba-8072-95debac20cdc None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:39.887590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97339e3c-97f8-42ba-8072-95debac20cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:39.887862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97339e3c-97f8-42ba-8072-95debac20cdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:39.900094 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97339e3c-97f8-42ba-8072-95debac20cdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-97339e3c-97f8-42ba-8072-95debac20cdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:02:39.905292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:39.907047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:39.909327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:39.909327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:39.909327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:39.909327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:39.909327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:02:39.909327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:02:39.909327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:02:39.909327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:02:39.909327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:02:39.909327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:02:39.909327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:02:39.909327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:39.909327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 572/1141] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:02:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:02:39.914675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93678edb-1922-4cac-a9ad-1381491aabfa None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:40.033944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93678edb-1922-4cac-a9ad-1381491aabfa None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:40.075165 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425957e-3414-4613-adfe-2495021a1a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:40.075986 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425957e-3414-4613-adfe-2495021a1a19 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:40.076357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425957e-3414-4613-adfe-2495021a1a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:40.076721 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425957e-3414-4613-adfe-2495021a1a19 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:40.077510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425957e-3414-4613-adfe-2495021a1a19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:40.077973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425957e-3414-4613-adfe-2495021a1a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:40.080256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425957e-3414-4613-adfe-2495021a1a19 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:40.092774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 570/1142] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:02:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:40.102397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97339e3c-97f8-42ba-8072-95debac20cdc None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:40.157888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:40.218448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97339e3c-97f8-42ba-8072-95debac20cdc None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:40.256693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68168d-ed6b-4434-936b-b0d1d5a3b2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:40.257336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68168d-ed6b-4434-936b-b0d1d5a3b2ff None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:40.257612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68168d-ed6b-4434-936b-b0d1d5a3b2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:40.257880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68168d-ed6b-4434-936b-b0d1d5a3b2ff None admin] PATH_INFO: `/v3/users/e4ec5392d0a14bc0bf8f4d689714806d` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:40.275011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68168d-ed6b-4434-936b-b0d1d5a3b2ff None admin] RBAC: Authorizing `identity:get_user(user_id=e4ec5392d0a14bc0bf8f4d689714806d)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:40.276664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68168d-ed6b-4434-936b-b0d1d5a3b2ff None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:40.278407 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 573/1143] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:02:40 2022] GET /identity/v3/users/e4ec5392d0a14bc0bf8f4d689714806d => generated 286 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:40.284804 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425957e-3414-4613-adfe-2495021a1a19 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:40.350226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:40.401865 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425957e-3414-4613-adfe-2495021a1a19 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:40.439029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e89cb9-6954-4ca1-82e3-f01190da20ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:40.439698 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e89cb9-6954-4ca1-82e3-f01190da20ff None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:40.439978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e89cb9-6954-4ca1-82e3-f01190da20ff None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:40.440249 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e89cb9-6954-4ca1-82e3-f01190da20ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:40.514591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e89cb9-6954-4ca1-82e3-f01190da20ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:40.516639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e89cb9-6954-4ca1-82e3-f01190da20ff None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:40.577750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:40.630831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 571/1144] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:02:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:02:40.640288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68168d-ed6b-4434-936b-b0d1d5a3b2ff None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:40.756311 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68168d-ed6b-4434-936b-b0d1d5a3b2ff None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:40.759660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:40.817745 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15ecaa-cd32-4de6-ba20-50a8087648fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:40.818852 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15ecaa-cd32-4de6-ba20-50a8087648fc None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:40.819487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15ecaa-cd32-4de6-ba20-50a8087648fc None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:40.819866 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15ecaa-cd32-4de6-ba20-50a8087648fc None admin] PATH_INFO: `/v3/projects/32e53c36052a47b89f3c651844d3bf84` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:40.821851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15ecaa-cd32-4de6-ba20-50a8087648fc None admin] RBAC: Authorizing `identity:get_project(project_id=32e53c36052a47b89f3c651844d3bf84)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:40.824607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15ecaa-cd32-4de6-ba20-50a8087648fc None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:40.828012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 574/1145] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:02:40 2022] GET /identity/v3/projects/32e53c36052a47b89f3c651844d3bf84 => generated 311 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:40.834891 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e89cb9-6954-4ca1-82e3-f01190da20ff None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:40.954271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e89cb9-6954-4ca1-82e3-f01190da20ff None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:40.998590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f173c33-6715-43c5-8717-338321e89e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:40.999248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f173c33-6715-43c5-8717-338321e89e5e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:40.999531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f173c33-6715-43c5-8717-338321e89e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:40.999801 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f173c33-6715-43c5-8717-338321e89e5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:41.000600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f173c33-6715-43c5-8717-338321e89e5e None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=32e53c36052a47b89f3c651844d3bf84, user.id=e4ec5392d0a14bc0bf8f4d689714806d) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:41.000911 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f173c33-6715-43c5-8717-338321e89e5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:41.002405 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f173c33-6715-43c5-8717-338321e89e5e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:41.009164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 572/1146] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:02:40 2022] GET /identity/v3/role_assignments?user.id=e4ec5392d0a14bc0bf8f4d689714806d&scope.project.id=32e53c36052a47b89f3c651844d3bf84&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:43.340897 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15d94-2a72-4537-aab7-a4508c98557b None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:43.341457 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15d94-2a72-4537-aab7-a4508c98557b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:43.341797 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a15d94-2a72-4537-aab7-a4508c98557b None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:43.344881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 575/1147] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:43.351033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034381b-0fd2-439c-8e5b-289c89de32b9 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:43.351630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034381b-0fd2-439c-8e5b-289c89de32b9 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:43.351630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034381b-0fd2-439c-8e5b-289c89de32b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:43.371096 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-1034381b-0fd2-439c-8e5b-289c89de32b9 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:43.372625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1034381b-0fd2-439c-8e5b-289c89de32b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:43.388672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:43.456175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 573/1148] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:43.734350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a15d94-2a72-4537-aab7-a4508c98557b None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:43.858444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a15d94-2a72-4537-aab7-a4508c98557b None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:43.901865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b0484-b29e-4044-b885-f14ddfb42226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:43.902515 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b0484-b29e-4044-b885-f14ddfb42226 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:43.902789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b0484-b29e-4044-b885-f14ddfb42226 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:43.903056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b0484-b29e-4044-b885-f14ddfb42226 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:43.914830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b0484-b29e-4044-b885-f14ddfb42226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-4a0b0484-b29e-4044-b885-f14ddfb42226 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:02:43.919883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:43.921143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:43.922152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:43.922152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:43.922152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:43.922152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:43.922152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:02:43.922152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:02:43.922152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:02:43.922152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:02:43.922152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:02:43.922152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:02:43.922152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:02:43.922152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:43.922152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 576/1149] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:02:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:43.930678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034381b-0fd2-439c-8e5b-289c89de32b9 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:44.058625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034381b-0fd2-439c-8e5b-289c89de32b9 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:44.102329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5eaa57-8812-4d59-a115-d8f7ffa43e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:44.102960 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eaa57-8812-4d59-a115-d8f7ffa43e75 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:44.103278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eaa57-8812-4d59-a115-d8f7ffa43e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:44.103551 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eaa57-8812-4d59-a115-d8f7ffa43e75 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:44.104240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eaa57-8812-4d59-a115-d8f7ffa43e75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:44.104672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eaa57-8812-4d59-a115-d8f7ffa43e75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:44.106240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eaa57-8812-4d59-a115-d8f7ffa43e75 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:44.120999 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 574/1150] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:02:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:44.128411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b0484-b29e-4044-b885-f14ddfb42226 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:44.184576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:44.311522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b0484-b29e-4044-b885-f14ddfb42226 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:44.352185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28ffec-aba8-45ce-9e66-fa12633a00b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:44.352909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28ffec-aba8-45ce-9e66-fa12633a00b0 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:44.353099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28ffec-aba8-45ce-9e66-fa12633a00b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:44.353368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28ffec-aba8-45ce-9e66-fa12633a00b0 None admin] PATH_INFO: `/v3/users/e4ec5392d0a14bc0bf8f4d689714806d` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:44.354652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28ffec-aba8-45ce-9e66-fa12633a00b0 None admin] RBAC: Authorizing `identity:get_user(user_id=e4ec5392d0a14bc0bf8f4d689714806d)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:44.356543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28ffec-aba8-45ce-9e66-fa12633a00b0 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:44.358584 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 577/1151] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:02:44 2022] GET /identity/v3/users/e4ec5392d0a14bc0bf8f4d689714806d => generated 286 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:02:44.364950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5eaa57-8812-4d59-a115-d8f7ffa43e75 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:44.379571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:44.479183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5eaa57-8812-4d59-a115-d8f7ffa43e75 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:44.523810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac72e8-c921-4205-8ee1-75ca2e8c3955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:44.524464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac72e8-c921-4205-8ee1-75ca2e8c3955 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:44.524744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac72e8-c921-4205-8ee1-75ca2e8c3955 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:44.525019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac72e8-c921-4205-8ee1-75ca2e8c3955 None admin] PATH_INFO: `/v3/projects/32e53c36052a47b89f3c651844d3bf84` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:44.526256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac72e8-c921-4205-8ee1-75ca2e8c3955 None admin] RBAC: Authorizing `identity:get_project(project_id=32e53c36052a47b89f3c651844d3bf84)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:44.527764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac72e8-c921-4205-8ee1-75ca2e8c3955 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:44.529514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 575/1152] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:02:44 2022] GET /identity/v3/projects/32e53c36052a47b89f3c651844d3bf84 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:44.535993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28ffec-aba8-45ce-9e66-fa12633a00b0 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:44.653608 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:44.657416 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28ffec-aba8-45ce-9e66-fa12633a00b0 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:44.698131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee721ba-4da1-422c-83e3-bd76f508e7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:44.698809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee721ba-4da1-422c-83e3-bd76f508e7a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:44.699085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee721ba-4da1-422c-83e3-bd76f508e7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:44.699387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee721ba-4da1-422c-83e3-bd76f508e7a5 None admin] PATH_INFO: `/v3/projects/32e53c36052a47b89f3c651844d3bf84/users/e4ec5392d0a14bc0bf8f4d689714806d/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:44.701390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee721ba-4da1-422c-83e3-bd76f508e7a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=32e53c36052a47b89f3c651844d3bf84, user_id=e4ec5392d0a14bc0bf8f4d689714806d, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:44.702877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee721ba-4da1-422c-83e3-bd76f508e7a5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:44.724055 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 578/1153] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:02:44 2022] PUT /identity/v3/projects/32e53c36052a47b89f3c651844d3bf84/users/e4ec5392d0a14bc0bf8f4d689714806d/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:02:47.073964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041e8c4-691f-4fcc-8c69-e03d3ad89272 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:47.073964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041e8c4-691f-4fcc-8c69-e03d3ad89272 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:47.073964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041e8c4-691f-4fcc-8c69-e03d3ad89272 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:47.075549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 576/1154] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:47.082318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986ef37-8d26-4583-85ac-b26e69b717e6 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:47.083085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986ef37-8d26-4583-85ac-b26e69b717e6 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:47.083085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986ef37-8d26-4583-85ac-b26e69b717e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:47.104857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-5986ef37-8d26-4583-85ac-b26e69b717e6 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:47.106094 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5986ef37-8d26-4583-85ac-b26e69b717e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:47.185210 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 579/1155] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:02:47.460369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041e8c4-691f-4fcc-8c69-e03d3ad89272 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:47.555477 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:47.583062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041e8c4-691f-4fcc-8c69-e03d3ad89272 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:47.625588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6694b-9299-484f-80e4-2de9c66e41b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:47.626247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6694b-9299-484f-80e4-2de9c66e41b0 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:47.626534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6694b-9299-484f-80e4-2de9c66e41b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:47.626813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6694b-9299-484f-80e4-2de9c66e41b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:47.640457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6694b-9299-484f-80e4-2de9c66e41b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-42e6694b-9299-484f-80e4-2de9c66e41b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:02:47.646091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:47.652052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:47.653993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:47.653993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:47.653993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:47.653993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:47.653993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:02:47.653993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:02:47.653993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:02:47.653993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:02:47.653993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:02:47.653993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:02:47.653993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:02:47.653993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:02:47.653993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 577/1156] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:02:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:47.655638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986ef37-8d26-4583-85ac-b26e69b717e6 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:47.763294 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:47.776312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986ef37-8d26-4583-85ac-b26e69b717e6 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:47.814007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb24c7a-8fe6-4216-b90d-71b7e29b3850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:47.814639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb24c7a-8fe6-4216-b90d-71b7e29b3850 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:47.814916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb24c7a-8fe6-4216-b90d-71b7e29b3850 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:47.815217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb24c7a-8fe6-4216-b90d-71b7e29b3850 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:47.815902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb24c7a-8fe6-4216-b90d-71b7e29b3850 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:47.816205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb24c7a-8fe6-4216-b90d-71b7e29b3850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:47.817626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb24c7a-8fe6-4216-b90d-71b7e29b3850 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:47.827474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 580/1157] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:02:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:47.835603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6694b-9299-484f-80e4-2de9c66e41b0 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:47.956743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6694b-9299-484f-80e4-2de9c66e41b0 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:47.995458 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309dced5-0f14-4c63-97d0-187fda94e315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:47.996114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309dced5-0f14-4c63-97d0-187fda94e315 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:47.996666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309dced5-0f14-4c63-97d0-187fda94e315 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:47.996939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309dced5-0f14-4c63-97d0-187fda94e315 None admin] PATH_INFO: `/v3/users/e4ec5392d0a14bc0bf8f4d689714806d` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:47.998146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309dced5-0f14-4c63-97d0-187fda94e315 None admin] RBAC: Authorizing `identity:get_user(user_id=e4ec5392d0a14bc0bf8f4d689714806d)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:47.999646 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309dced5-0f14-4c63-97d0-187fda94e315 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:48.001350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 578/1158] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:02:47 2022] GET /identity/v3/users/e4ec5392d0a14bc0bf8f4d689714806d => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:48.007656 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb24c7a-8fe6-4216-b90d-71b7e29b3850 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:48.149564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb24c7a-8fe6-4216-b90d-71b7e29b3850 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:48.207527 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21dfbd-8332-4af9-94d0-788d80f178df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:48.208167 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21dfbd-8332-4af9-94d0-788d80f178df None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:48.208442 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21dfbd-8332-4af9-94d0-788d80f178df None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:48.208709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21dfbd-8332-4af9-94d0-788d80f178df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:48.233285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:48.287805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21dfbd-8332-4af9-94d0-788d80f178df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:48.289356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21dfbd-8332-4af9-94d0-788d80f178df None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:48.402907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 581/1159] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:02:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 13:02:48.410018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309dced5-0f14-4c63-97d0-187fda94e315 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:48.421392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:48.527776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309dced5-0f14-4c63-97d0-187fda94e315 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:48.586941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fec40-5062-4dfc-8820-5a2b446819d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:48.588021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fec40-5062-4dfc-8820-5a2b446819d2 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:48.588516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fec40-5062-4dfc-8820-5a2b446819d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:48.588987 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fec40-5062-4dfc-8820-5a2b446819d2 None admin] PATH_INFO: `/v3/projects/32e53c36052a47b89f3c651844d3bf84` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:48.591096 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fec40-5062-4dfc-8820-5a2b446819d2 None admin] RBAC: Authorizing `identity:get_project(project_id=32e53c36052a47b89f3c651844d3bf84)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:48.593841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fec40-5062-4dfc-8820-5a2b446819d2 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:48.596963 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 579/1160] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:02:48 2022] GET /identity/v3/projects/32e53c36052a47b89f3c651844d3bf84 => generated 311 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:48.604281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21dfbd-8332-4af9-94d0-788d80f178df None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:48.667730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:48.736726 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21dfbd-8332-4af9-94d0-788d80f178df None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:48.778536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351544b1-ebd3-4d8c-88c2-bc4390bbe2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:48.779252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351544b1-ebd3-4d8c-88c2-bc4390bbe2ac None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:48.779580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351544b1-ebd3-4d8c-88c2-bc4390bbe2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:48.779898 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351544b1-ebd3-4d8c-88c2-bc4390bbe2ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:48.780698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351544b1-ebd3-4d8c-88c2-bc4390bbe2ac None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=32e53c36052a47b89f3c651844d3bf84, user.id=e4ec5392d0a14bc0bf8f4d689714806d) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:48.781077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351544b1-ebd3-4d8c-88c2-bc4390bbe2ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:48.782610 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351544b1-ebd3-4d8c-88c2-bc4390bbe2ac None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:48.789718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 582/1161] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:02:48 2022] GET /identity/v3/role_assignments?user.id=e4ec5392d0a14bc0bf8f4d689714806d&scope.project.id=32e53c36052a47b89f3c651844d3bf84&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:51.162644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104891eb-115f-4e0a-99dd-16be1c5a3b2e None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:51.163278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104891eb-115f-4e0a-99dd-16be1c5a3b2e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:51.163278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104891eb-115f-4e0a-99dd-16be1c5a3b2e None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:51.164818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 580/1162] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:02:51.171886 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05490eff-5f38-42f7-8e34-a8dc2ecdb04c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:51.172483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05490eff-5f38-42f7-8e34-a8dc2ecdb04c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:51.172483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05490eff-5f38-42f7-8e34-a8dc2ecdb04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:51.192934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-05490eff-5f38-42f7-8e34-a8dc2ecdb04c None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:51.194154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-05490eff-5f38-42f7-8e34-a8dc2ecdb04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:51.273416 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 583/1163] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:51.556718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104891eb-115f-4e0a-99dd-16be1c5a3b2e None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:51.647229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:51.693905 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104891eb-115f-4e0a-99dd-16be1c5a3b2e None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:51.744253 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa9032-6d72-4636-af27-bcf127255889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:51.744882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa9032-6d72-4636-af27-bcf127255889 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:51.745186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa9032-6d72-4636-af27-bcf127255889 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:51.745442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa9032-6d72-4636-af27-bcf127255889 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:51.827084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa9032-6d72-4636-af27-bcf127255889 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:51.828734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa9032-6d72-4636-af27-bcf127255889 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:51.913549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:51.943647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 581/1164] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:02:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:02:51.949993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05490eff-5f38-42f7-8e34-a8dc2ecdb04c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:52.005125 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:52.081124 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05490eff-5f38-42f7-8e34-a8dc2ecdb04c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:52.129392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c35c45-5003-4faf-85f8-6e502ae579a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:52.130070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c35c45-5003-4faf-85f8-6e502ae579a5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:52.130342 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c35c45-5003-4faf-85f8-6e502ae579a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:52.130609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c35c45-5003-4faf-85f8-6e502ae579a5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:52.140125 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c35c45-5003-4faf-85f8-6e502ae579a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:52.141661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c35c45-5003-4faf-85f8-6e502ae579a5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-28c35c45-5003-4faf-85f8-6e502ae579a5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:02:52.156415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:02:52.158497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:02:52.160501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 584/1165] 10.222.0.61 () {56 vars in 1117 bytes} [Wed Sep 7 13:02:51 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:02:52.166035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa9032-6d72-4636-af27-bcf127255889 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:52.285173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa9032-6d72-4636-af27-bcf127255889 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:52.323303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad7c79-f950-48c1-8269-7aeddcdbe268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:52.323933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad7c79-f950-48c1-8269-7aeddcdbe268 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:52.324212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad7c79-f950-48c1-8269-7aeddcdbe268 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:52.324478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad7c79-f950-48c1-8269-7aeddcdbe268 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:52.325141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad7c79-f950-48c1-8269-7aeddcdbe268 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:02:52.325442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad7c79-f950-48c1-8269-7aeddcdbe268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:52.326827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad7c79-f950-48c1-8269-7aeddcdbe268 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:52.336065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 582/1166] 10.222.0.61 () {56 vars in 1115 bytes} [Wed Sep 7 13:02:52 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:54.642014 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29034a-9dba-4146-a3e8-e4300e398893 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:54.642583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29034a-9dba-4146-a3e8-e4300e398893 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:54.642583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29034a-9dba-4146-a3e8-e4300e398893 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:54.643954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 585/1167] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:02:54.649922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80493f29-ad43-453e-8f69-7aa6e39bd884 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:54.650442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80493f29-ad43-453e-8f69-7aa6e39bd884 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:54.650442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80493f29-ad43-453e-8f69-7aa6e39bd884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:54.670216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-80493f29-ad43-453e-8f69-7aa6e39bd884 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:54.672015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-80493f29-ad43-453e-8f69-7aa6e39bd884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:54.708169 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:54.752420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 583/1168] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:02:55.030322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29034a-9dba-4146-a3e8-e4300e398893 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:55.152926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29034a-9dba-4146-a3e8-e4300e398893 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:55.162687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:55.195648 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13760b17-c8e3-4fc8-9fc6-6089d1a3522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:55.196267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13760b17-c8e3-4fc8-9fc6-6089d1a3522d None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:55.196539 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13760b17-c8e3-4fc8-9fc6-6089d1a3522d None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:55.196805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13760b17-c8e3-4fc8-9fc6-6089d1a3522d None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:55.197484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13760b17-c8e3-4fc8-9fc6-6089d1a3522d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:55.198893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13760b17-c8e3-4fc8-9fc6-6089d1a3522d None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:55.217749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 586/1169] 10.222.0.61 () {60 vars in 1138 bytes} [Wed Sep 7 13:02:55 2022] POST /identity/v3/domains => generated 267 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:02:57.551948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f85d14-d9d3-4dfc-ac08-b1d6e9bfa073 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:57.552512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f85d14-d9d3-4dfc-ac08-b1d6e9bfa073 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:57.552620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f85d14-d9d3-4dfc-ac08-b1d6e9bfa073 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:57.554040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 584/1170] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:02:57.560513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c695c-6809-4f23-ba86-a4985994257e None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:57.561056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c695c-6809-4f23-ba86-a4985994257e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:57.561216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c695c-6809-4f23-ba86-a4985994257e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:57.582383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-db1c695c-6809-4f23-ba86-a4985994257e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:02:57.583714 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-db1c695c-6809-4f23-ba86-a4985994257e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:02:57.662186 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 587/1171] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:02:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:02:57.938420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f85d14-d9d3-4dfc-ac08-b1d6e9bfa073 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:58.057952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f85d14-d9d3-4dfc-ac08-b1d6e9bfa073 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:58.099677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcbfb2-e005-414a-80f3-0e96b3cdc43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:58.100338 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcbfb2-e005-414a-80f3-0e96b3cdc43d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:58.100617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcbfb2-e005-414a-80f3-0e96b3cdc43d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:58.100898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcbfb2-e005-414a-80f3-0e96b3cdc43d None admin] PATH_INFO: `/v3/domains/61c6e70099f543f780f528b2513830db` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:58.114121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcbfb2-e005-414a-80f3-0e96b3cdc43d None admin] RBAC: Authorizing `identity:get_domain(domain_id=61c6e70099f543f780f528b2513830db)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:58.115773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcbfb2-e005-414a-80f3-0e96b3cdc43d None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:58.117674 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 585/1172] 10.222.0.61 () {56 vars in 1183 bytes} [Wed Sep 7 13:02:57 2022] GET /identity/v3/domains/61c6e70099f543f780f528b2513830db => generated 267 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:02:58.124668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c695c-6809-4f23-ba86-a4985994257e None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:02:58.236116 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:02:58.243582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c695c-6809-4f23-ba86-a4985994257e None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:02:58.284266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50147d7-3480-4303-997b-397fe6a5b100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:02:58.284978 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50147d7-3480-4303-997b-397fe6a5b100 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:02:58.285305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50147d7-3480-4303-997b-397fe6a5b100 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:02:58.285625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50147d7-3480-4303-997b-397fe6a5b100 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:02:58.286421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50147d7-3480-4303-997b-397fe6a5b100 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:02:58.287992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50147d7-3480-4303-997b-397fe6a5b100 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:02:58.319776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 588/1173] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 13:02:58 2022] POST /identity/v3/projects => generated 361 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:03:00.740498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50e779-6cca-4f68-a7e7-c32e5069e86d None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:00.741118 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50e779-6cca-4f68-a7e7-c32e5069e86d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:00.741118 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50e779-6cca-4f68-a7e7-c32e5069e86d None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:00.742661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 586/1174] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:03:00.749387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9606c-726b-46fb-b913-ba661fb7fb24 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:00.749933 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9606c-726b-46fb-b913-ba661fb7fb24 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:00.749933 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9606c-726b-46fb-b913-ba661fb7fb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:00.770459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-cbf9606c-726b-46fb-b913-ba661fb7fb24 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:03:00.771749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-cbf9606c-726b-46fb-b913-ba661fb7fb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:03:00.851529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 589/1175] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:03:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:03:01.131973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50e779-6cca-4f68-a7e7-c32e5069e86d None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:01.163418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:01.248940 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50e779-6cca-4f68-a7e7-c32e5069e86d None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:01.285310 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1158305-313d-494d-ac10-8dc1f4b12cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:01.285950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1158305-313d-494d-ac10-8dc1f4b12cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:01.286231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1158305-313d-494d-ac10-8dc1f4b12cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:01.286504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1158305-313d-494d-ac10-8dc1f4b12cd9 None admin] PATH_INFO: `/v3/domains/61c6e70099f543f780f528b2513830db` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:01.287818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1158305-313d-494d-ac10-8dc1f4b12cd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=61c6e70099f543f780f528b2513830db)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:01.289273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1158305-313d-494d-ac10-8dc1f4b12cd9 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:01.290915 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 587/1176] 10.222.0.61 () {56 vars in 1183 bytes} [Wed Sep 7 13:03:01 2022] GET /identity/v3/domains/61c6e70099f543f780f528b2513830db => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:03:01.297790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9606c-726b-46fb-b913-ba661fb7fb24 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:01.384881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:01.411360 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9606c-726b-46fb-b913-ba661fb7fb24 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:01.450515 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87984392-9d3c-4771-8d1f-cde6e9f8a1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:01.451162 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87984392-9d3c-4771-8d1f-cde6e9f8a1f8 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:01.451442 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87984392-9d3c-4771-8d1f-cde6e9f8a1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:01.451709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87984392-9d3c-4771-8d1f-cde6e9f8a1f8 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:01.452929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87984392-9d3c-4771-8d1f-cde6e9f8a1f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:01.454409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87984392-9d3c-4771-8d1f-cde6e9f8a1f8 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:01.482119 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 590/1177] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 13:03:01 2022] POST /identity/v3/users => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:03:05.268839 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bab2d-26d7-4d12-9fbf-7bb0e0f53cf3 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:05.269461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bab2d-26d7-4d12-9fbf-7bb0e0f53cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:05.269461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bab2d-26d7-4d12-9fbf-7bb0e0f53cf3 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:05.270887 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 588/1178] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:03:05.278052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453e48d-5fc4-48a0-b933-8ff69bdd0f0a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:05.278639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453e48d-5fc4-48a0-b933-8ff69bdd0f0a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:05.279162 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453e48d-5fc4-48a0-b933-8ff69bdd0f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:05.313735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-3453e48d-5fc4-48a0-b933-8ff69bdd0f0a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:03:05.315861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3453e48d-5fc4-48a0-b933-8ff69bdd0f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:03:05.413965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 591/1179] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:03:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:03:05.713174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8bab2d-26d7-4d12-9fbf-7bb0e0f53cf3 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:05.826050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:05.835618 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8bab2d-26d7-4d12-9fbf-7bb0e0f53cf3 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:05.874833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d492669-ee33-465c-8fab-5e98ed965f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:05.875520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d492669-ee33-465c-8fab-5e98ed965f05 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:05.875805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d492669-ee33-465c-8fab-5e98ed965f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:05.876080 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d492669-ee33-465c-8fab-5e98ed965f05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:05.888693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d492669-ee33-465c-8fab-5e98ed965f05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-0d492669-ee33-465c-8fab-5e98ed965f05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:03:05.894275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:03:05.895296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:03:05.896155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:03:05.896155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:03:05.896155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:03:05.896155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:03:05.896155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:03:05.896155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:03:05.896155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:03:05.896155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:03:05.896155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:03:05.896155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:03:05.896155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:03:05.896155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:03:05.896155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 589/1180] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:03:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:03:05.902048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453e48d-5fc4-48a0-b933-8ff69bdd0f0a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:05.974853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:06.020623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453e48d-5fc4-48a0-b933-8ff69bdd0f0a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:06.059872 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79da75b-5aa2-4131-8f69-3e03ae10dfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:06.060494 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79da75b-5aa2-4131-8f69-3e03ae10dfa6 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:06.061388 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79da75b-5aa2-4131-8f69-3e03ae10dfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:06.061664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79da75b-5aa2-4131-8f69-3e03ae10dfa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:06.062344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79da75b-5aa2-4131-8f69-3e03ae10dfa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:03:06.062644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79da75b-5aa2-4131-8f69-3e03ae10dfa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:06.064109 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79da75b-5aa2-4131-8f69-3e03ae10dfa6 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:06.074431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 592/1181] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:03:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:03:06.081669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d492669-ee33-465c-8fab-5e98ed965f05 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:06.273816 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d492669-ee33-465c-8fab-5e98ed965f05 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:06.333573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d325b22-5512-4b73-aa3e-b7c1248e4175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:06.334231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d325b22-5512-4b73-aa3e-b7c1248e4175 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:06.334518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d325b22-5512-4b73-aa3e-b7c1248e4175 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:06.334795 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d325b22-5512-4b73-aa3e-b7c1248e4175 None admin] PATH_INFO: `/v3/users/0127611ebdda4d78a8cc5445e93f4549` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:06.350498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d325b22-5512-4b73-aa3e-b7c1248e4175 None admin] RBAC: Authorizing `identity:get_user(user_id=0127611ebdda4d78a8cc5445e93f4549)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:06.352036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d325b22-5512-4b73-aa3e-b7c1248e4175 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:06.353805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 590/1182] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:03:06 2022] GET /identity/v3/users/0127611ebdda4d78a8cc5445e93f4549 => generated 311 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:03:06.360928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79da75b-5aa2-4131-8f69-3e03ae10dfa6 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:06.486068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79da75b-5aa2-4131-8f69-3e03ae10dfa6 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:06.533156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:06.547876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23d9cf-a434-4ff0-8767-7fa3f21f4a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:06.548787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23d9cf-a434-4ff0-8767-7fa3f21f4a43 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:06.549281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23d9cf-a434-4ff0-8767-7fa3f21f4a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:06.549759 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23d9cf-a434-4ff0-8767-7fa3f21f4a43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:06.644803 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23d9cf-a434-4ff0-8767-7fa3f21f4a43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:06.646366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23d9cf-a434-4ff0-8767-7fa3f21f4a43 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:06.761109 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 593/1183] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:03:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:03:06.768424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d325b22-5512-4b73-aa3e-b7c1248e4175 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:06.786907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:06.883499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d325b22-5512-4b73-aa3e-b7c1248e4175 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:06.922618 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6727ad-577c-42b3-9b0c-e9232cfd24bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:06.923306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6727ad-577c-42b3-9b0c-e9232cfd24bb None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:06.923590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6727ad-577c-42b3-9b0c-e9232cfd24bb None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:06.923871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6727ad-577c-42b3-9b0c-e9232cfd24bb None admin] PATH_INFO: `/v3/projects/ca005307e1de41dc89ae157bd45eecd9` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:06.925064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6727ad-577c-42b3-9b0c-e9232cfd24bb None admin] RBAC: Authorizing `identity:get_project(project_id=ca005307e1de41dc89ae157bd45eecd9)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:06.926519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6727ad-577c-42b3-9b0c-e9232cfd24bb None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:06.928255 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 591/1184] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:03:06 2022] GET /identity/v3/projects/ca005307e1de41dc89ae157bd45eecd9 => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:03:06.934832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23d9cf-a434-4ff0-8767-7fa3f21f4a43 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:06.966239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:07.061602 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23d9cf-a434-4ff0-8767-7fa3f21f4a43 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:07.100635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56177e2-c3c7-4dc1-b09d-a6aaccaa69a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:07.101255 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56177e2-c3c7-4dc1-b09d-a6aaccaa69a6 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:07.101534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56177e2-c3c7-4dc1-b09d-a6aaccaa69a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:07.101806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56177e2-c3c7-4dc1-b09d-a6aaccaa69a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:07.102551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56177e2-c3c7-4dc1-b09d-a6aaccaa69a6 None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=ca005307e1de41dc89ae157bd45eecd9, user.id=0127611ebdda4d78a8cc5445e93f4549) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:03:07.102861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56177e2-c3c7-4dc1-b09d-a6aaccaa69a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:07.104382 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56177e2-c3c7-4dc1-b09d-a6aaccaa69a6 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:07.111414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 594/1185] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:03:06 2022] GET /identity/v3/role_assignments?user.id=0127611ebdda4d78a8cc5445e93f4549&scope.project.id=ca005307e1de41dc89ae157bd45eecd9&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:03:09.339200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbff61e-235d-4c0e-8a6a-60059f976927 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:09.339762 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbff61e-235d-4c0e-8a6a-60059f976927 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:09.339762 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbff61e-235d-4c0e-8a6a-60059f976927 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:09.341197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 592/1186] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:03:09.347132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c2a5c-0240-439e-82b8-a095c5ec5cdb None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:09.347697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c2a5c-0240-439e-82b8-a095c5ec5cdb None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:09.347821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c2a5c-0240-439e-82b8-a095c5ec5cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:09.366618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-2d9c2a5c-0240-439e-82b8-a095c5ec5cdb None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:03:09.367921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2d9c2a5c-0240-439e-82b8-a095c5ec5cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:03:09.441755 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 595/1187] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:03:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:03:09.712965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbff61e-235d-4c0e-8a6a-60059f976927 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:09.807357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:09.829833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbff61e-235d-4c0e-8a6a-60059f976927 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:09.870652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433840b-5455-4157-910d-1d65cc1e1c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:09.871577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433840b-5455-4157-910d-1d65cc1e1c17 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:09.871936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433840b-5455-4157-910d-1d65cc1e1c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:09.872121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433840b-5455-4157-910d-1d65cc1e1c17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:09.884765 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433840b-5455-4157-910d-1d65cc1e1c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-b433840b-5455-4157-910d-1d65cc1e1c17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:03:09.889855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:03:09.891764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:03:09.893669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:03:09.893669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:03:09.893669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:03:09.893669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:03:09.893669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:03:09.893669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:03:09.893669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:03:09.893669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:03:09.893669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:03:09.893669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:03:09.893669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:03:09.893669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:03:09.893669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 593/1188] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:03:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:03:09.899202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c2a5c-0240-439e-82b8-a095c5ec5cdb None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:09.923977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:10.018629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c2a5c-0240-439e-82b8-a095c5ec5cdb None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:10.060220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d511a8-b2e1-491d-a4f8-65c578de1074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:10.061025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d511a8-b2e1-491d-a4f8-65c578de1074 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:10.061389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d511a8-b2e1-491d-a4f8-65c578de1074 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:10.061754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d511a8-b2e1-491d-a4f8-65c578de1074 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:10.062536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d511a8-b2e1-491d-a4f8-65c578de1074 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:03:10.062934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d511a8-b2e1-491d-a4f8-65c578de1074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:10.064488 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d511a8-b2e1-491d-a4f8-65c578de1074 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:10.076740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 596/1189] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:03:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:03:10.083494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433840b-5455-4157-910d-1d65cc1e1c17 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:10.199371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433840b-5455-4157-910d-1d65cc1e1c17 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:10.239861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f78149-2549-4622-be08-cf0e4f3a08ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:10.240513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f78149-2549-4622-be08-cf0e4f3a08ab None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:10.240790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f78149-2549-4622-be08-cf0e4f3a08ab None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:10.241062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f78149-2549-4622-be08-cf0e4f3a08ab None admin] PATH_INFO: `/v3/users/0127611ebdda4d78a8cc5445e93f4549` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:10.242240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f78149-2549-4622-be08-cf0e4f3a08ab None admin] RBAC: Authorizing `identity:get_user(user_id=0127611ebdda4d78a8cc5445e93f4549)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:10.243738 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f78149-2549-4622-be08-cf0e4f3a08ab None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:10.245697 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 594/1190] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:03:10 2022] GET /identity/v3/users/0127611ebdda4d78a8cc5445e93f4549 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:03:10.252453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d511a8-b2e1-491d-a4f8-65c578de1074 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:10.345762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:10.367513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d511a8-b2e1-491d-a4f8-65c578de1074 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:10.406690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc700a4-97d6-4cfd-b84a-23a3d04b74a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:10.407433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc700a4-97d6-4cfd-b84a-23a3d04b74a3 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:10.407757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc700a4-97d6-4cfd-b84a-23a3d04b74a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:10.408074 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc700a4-97d6-4cfd-b84a-23a3d04b74a3 None admin] PATH_INFO: `/v3/projects/ca005307e1de41dc89ae157bd45eecd9` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:10.409599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc700a4-97d6-4cfd-b84a-23a3d04b74a3 None admin] RBAC: Authorizing `identity:get_project(project_id=ca005307e1de41dc89ae157bd45eecd9)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:10.411187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc700a4-97d6-4cfd-b84a-23a3d04b74a3 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:10.413101 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 597/1191] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:03:10 2022] GET /identity/v3/projects/ca005307e1de41dc89ae157bd45eecd9 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:03:10.419662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f78149-2549-4622-be08-cf0e4f3a08ab None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:10.436922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:10.540094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f78149-2549-4622-be08-cf0e4f3a08ab None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:10.584261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c11cb5-56cb-45bb-8262-766a7cceb7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:10.585426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c11cb5-56cb-45bb-8262-766a7cceb7be None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:10.586034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c11cb5-56cb-45bb-8262-766a7cceb7be None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:10.586586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c11cb5-56cb-45bb-8262-766a7cceb7be None admin] PATH_INFO: `/v3/projects/ca005307e1de41dc89ae157bd45eecd9/users/0127611ebdda4d78a8cc5445e93f4549/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:10.590067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c11cb5-56cb-45bb-8262-766a7cceb7be None admin] RBAC: Authorizing `identity:create_grant(project_id=ca005307e1de41dc89ae157bd45eecd9, user_id=0127611ebdda4d78a8cc5445e93f4549, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:10.593482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c11cb5-56cb-45bb-8262-766a7cceb7be None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:10.616123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 595/1192] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:03:10 2022] PUT /identity/v3/projects/ca005307e1de41dc89ae157bd45eecd9/users/0127611ebdda4d78a8cc5445e93f4549/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:03:12.960003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f163549-68ec-46c3-905b-bd7d3ce345ff None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:12.960906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f163549-68ec-46c3-905b-bd7d3ce345ff None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:12.960906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f163549-68ec-46c3-905b-bd7d3ce345ff None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:12.962434 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 598/1193] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:03:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:03:12.969142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef9b3e-2b52-4d2f-8f63-a937a6b4a106 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:12.969743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef9b3e-2b52-4d2f-8f63-a937a6b4a106 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:12.969743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef9b3e-2b52-4d2f-8f63-a937a6b4a106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:12.990162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-deef9b3e-2b52-4d2f-8f63-a937a6b4a106 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:03:12.991422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-deef9b3e-2b52-4d2f-8f63-a937a6b4a106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:03:13.063609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:13.070086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 596/1194] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:03:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:03:13.349710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f163549-68ec-46c3-905b-bd7d3ce345ff None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:13.468672 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f163549-68ec-46c3-905b-bd7d3ce345ff None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:13.506959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030669b-b91f-4b68-935b-bd2517d1cfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:13.507647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030669b-b91f-4b68-935b-bd2517d1cfbc None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:13.507924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030669b-b91f-4b68-935b-bd2517d1cfbc None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:13.508195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030669b-b91f-4b68-935b-bd2517d1cfbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:13.521086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1030669b-b91f-4b68-935b-bd2517d1cfbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-1030669b-b91f-4b68-935b-bd2517d1cfbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:03:13.526067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:03:13.527913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:03:13.530547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:03:13.530547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:03:13.530547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:03:13.530547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:03:13.530547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:03:13.530547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:03:13.530547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:03:13.530547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:03:13.530547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:03:13.530547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:03:13.530547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:03:13.530547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:03:13.530547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 599/1195] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:03:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:03:13.535944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef9b3e-2b52-4d2f-8f63-a937a6b4a106 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:13.646185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef9b3e-2b52-4d2f-8f63-a937a6b4a106 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:13.688961 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977b8ed-66e3-41c4-ac6f-4fd382c28a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:13.689886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977b8ed-66e3-41c4-ac6f-4fd382c28a13 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:13.690324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977b8ed-66e3-41c4-ac6f-4fd382c28a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:13.690779 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977b8ed-66e3-41c4-ac6f-4fd382c28a13 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:13.691943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977b8ed-66e3-41c4-ac6f-4fd382c28a13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:03:13.692268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977b8ed-66e3-41c4-ac6f-4fd382c28a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:13.693692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977b8ed-66e3-41c4-ac6f-4fd382c28a13 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:13.703431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 597/1196] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:03:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:03:13.712875 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030669b-b91f-4b68-935b-bd2517d1cfbc None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:13.728635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:13.832507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030669b-b91f-4b68-935b-bd2517d1cfbc None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:13.873316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec511fe-d865-41ba-8fc2-1888b0569a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:13.873968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec511fe-d865-41ba-8fc2-1888b0569a41 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:13.874250 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec511fe-d865-41ba-8fc2-1888b0569a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:13.874522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec511fe-d865-41ba-8fc2-1888b0569a41 None admin] PATH_INFO: `/v3/users/0127611ebdda4d78a8cc5445e93f4549` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:13.875756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec511fe-d865-41ba-8fc2-1888b0569a41 None admin] RBAC: Authorizing `identity:get_user(user_id=0127611ebdda4d78a8cc5445e93f4549)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:13.877227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec511fe-d865-41ba-8fc2-1888b0569a41 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:13.878895 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 600/1197] 10.222.0.61 () {56 vars in 1177 bytes} [Wed Sep 7 13:03:13 2022] GET /identity/v3/users/0127611ebdda4d78a8cc5445e93f4549 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:03:13.885154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977b8ed-66e3-41c4-ac6f-4fd382c28a13 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:13.958240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:14.002895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977b8ed-66e3-41c4-ac6f-4fd382c28a13 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:14.045201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fccfba-122c-46ad-85ed-7cb4bf5dc859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:14.045835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fccfba-122c-46ad-85ed-7cb4bf5dc859 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:14.046117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fccfba-122c-46ad-85ed-7cb4bf5dc859 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:14.046388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fccfba-122c-46ad-85ed-7cb4bf5dc859 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:14.128591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fccfba-122c-46ad-85ed-7cb4bf5dc859 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:14.130069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fccfba-122c-46ad-85ed-7cb4bf5dc859 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:14.199396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:14.249994 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 598/1198] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:03:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:03:14.259670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec511fe-d865-41ba-8fc2-1888b0569a41 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:14.350503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:14.375279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec511fe-d865-41ba-8fc2-1888b0569a41 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:14.413435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7149c2e-0282-4ed9-aa38-3878c1abb4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:14.414089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7149c2e-0282-4ed9-aa38-3878c1abb4db None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:14.414370 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7149c2e-0282-4ed9-aa38-3878c1abb4db None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:14.414663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7149c2e-0282-4ed9-aa38-3878c1abb4db None admin] PATH_INFO: `/v3/projects/ca005307e1de41dc89ae157bd45eecd9` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:14.415839 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7149c2e-0282-4ed9-aa38-3878c1abb4db None admin] RBAC: Authorizing `identity:get_project(project_id=ca005307e1de41dc89ae157bd45eecd9)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:14.417829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7149c2e-0282-4ed9-aa38-3878c1abb4db None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:14.419697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 601/1199] 10.222.0.61 () {56 vars in 1186 bytes} [Wed Sep 7 13:03:14 2022] GET /identity/v3/projects/ca005307e1de41dc89ae157bd45eecd9 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:03:14.426699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fccfba-122c-46ad-85ed-7cb4bf5dc859 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:03:14.535637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fccfba-122c-46ad-85ed-7cb4bf5dc859 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:03:14.574359 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d43d0a-7f4e-4cc5-9edf-5563cccb17e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:03:14.574962 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d43d0a-7f4e-4cc5-9edf-5563cccb17e8 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:03:14.575271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d43d0a-7f4e-4cc5-9edf-5563cccb17e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:03:14.575543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d43d0a-7f4e-4cc5-9edf-5563cccb17e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:03:14.576840 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d43d0a-7f4e-4cc5-9edf-5563cccb17e8 None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=ca005307e1de41dc89ae157bd45eecd9, user.id=0127611ebdda4d78a8cc5445e93f4549) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:03:14.577144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d43d0a-7f4e-4cc5-9edf-5563cccb17e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:03:14.578591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d43d0a-7f4e-4cc5-9edf-5563cccb17e8 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:03:14.585379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 599/1200] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:03:14 2022] GET /identity/v3/role_assignments?user.id=0127611ebdda4d78a8cc5445e93f4549&scope.project.id=ca005307e1de41dc89ae157bd45eecd9&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:03:24.409215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=27 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:03:24.583472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=87 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:18.138562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db411b-50b1-4a48-94c1-8cebfe5b1614 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:18.139541 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db411b-50b1-4a48-94c1-8cebfe5b1614 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:18.139541 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db411b-50b1-4a48-94c1-8cebfe5b1614 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:18.140824 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 602/1201] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:04:18.148080 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581e2d7-69e4-4c1f-aa3a-64d6faf60035 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:18.149034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581e2d7-69e4-4c1f-aa3a-64d6faf60035 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:18.149034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581e2d7-69e4-4c1f-aa3a-64d6faf60035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:18.170474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-1581e2d7-69e4-4c1f-aa3a-64d6faf60035 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:18.172000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1581e2d7-69e4-4c1f-aa3a-64d6faf60035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:18.310364 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 600/1202] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:04:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:04:18.709702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db411b-50b1-4a48-94c1-8cebfe5b1614 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:18.890172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db411b-50b1-4a48-94c1-8cebfe5b1614 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:18.956378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f4f50-68ff-4ea3-a8dd-37643bbcf8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:18.957106 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f4f50-68ff-4ea3-a8dd-37643bbcf8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:18.957392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f4f50-68ff-4ea3-a8dd-37643bbcf8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:18.957716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f4f50-68ff-4ea3-a8dd-37643bbcf8b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:18.974154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f4f50-68ff-4ea3-a8dd-37643bbcf8b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:18.976239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f4f50-68ff-4ea3-a8dd-37643bbcf8b1 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-415f4f50-68ff-4ea3-a8dd-37643bbcf8b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:04:18.999235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:19.000362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:04:19.001383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:04:19.001383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:19.001383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:19.001383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 603/1203] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:04:19.011127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581e2d7-69e4-4c1f-aa3a-64d6faf60035 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:19.191162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581e2d7-69e4-4c1f-aa3a-64d6faf60035 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:19.246101 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:19.262371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f880a-6dad-4bce-826a-9e9903f3e139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:19.263352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f880a-6dad-4bce-826a-9e9903f3e139 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:19.265274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f880a-6dad-4bce-826a-9e9903f3e139 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:19.265849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f880a-6dad-4bce-826a-9e9903f3e139 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:19.267161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f880a-6dad-4bce-826a-9e9903f3e139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:19.267722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f880a-6dad-4bce-826a-9e9903f3e139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:19.272330 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f880a-6dad-4bce-826a-9e9903f3e139 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:19.315817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 601/1204] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:19.325469 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f4f50-68ff-4ea3-a8dd-37643bbcf8b1 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:19.459607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:19.573477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f4f50-68ff-4ea3-a8dd-37643bbcf8b1 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:19.642460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876fa23-2253-4c5c-a687-5d1b27c257bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:19.643530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876fa23-2253-4c5c-a687-5d1b27c257bb None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:19.643979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876fa23-2253-4c5c-a687-5d1b27c257bb None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:19.644420 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876fa23-2253-4c5c-a687-5d1b27c257bb None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:19.645251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876fa23-2253-4c5c-a687-5d1b27c257bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:19.647004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876fa23-2253-4c5c-a687-5d1b27c257bb None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:19.678066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 604/1205] 10.222.0.61 () {60 vars in 1132 bytes} [Wed Sep 7 13:04:19 2022] POST /identity/v3/users => generated 251 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:04:23.231402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423eb7d-e7a2-494b-ba9e-6cfb06a67f7a None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:23.232150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423eb7d-e7a2-494b-ba9e-6cfb06a67f7a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:23.232150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423eb7d-e7a2-494b-ba9e-6cfb06a67f7a None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:23.233715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 602/1206] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:04:23.240161 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60311f-6939-4a9c-9b55-531d6176b8da None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:23.241160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60311f-6939-4a9c-9b55-531d6176b8da None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:23.241160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60311f-6939-4a9c-9b55-531d6176b8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:23.265563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-5e60311f-6939-4a9c-9b55-531d6176b8da None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:23.266870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5e60311f-6939-4a9c-9b55-531d6176b8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:23.380209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 605/1207] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:04:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:04:23.886009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423eb7d-e7a2-494b-ba9e-6cfb06a67f7a None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:24.151424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423eb7d-e7a2-494b-ba9e-6cfb06a67f7a None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:24.236073 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8bc74-114e-4c3c-b9d7-bd0f644ca77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:24.237191 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8bc74-114e-4c3c-b9d7-bd0f644ca77d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:24.237667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8bc74-114e-4c3c-b9d7-bd0f644ca77d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:24.238162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8bc74-114e-4c3c-b9d7-bd0f644ca77d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:24.260728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8bc74-114e-4c3c-b9d7-bd0f644ca77d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-ffb8bc74-114e-4c3c-b9d7-bd0f644ca77d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:04:24.271172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:24.273829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:24.276420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:24.276420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:24.276420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:24.276420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:24.276420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:04:24.276420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:04:24.276420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:04:24.276420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:04:24.276420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:04:24.276420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:04:24.276420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:04:24.276420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:24.284543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60311f-6939-4a9c-9b55-531d6176b8da None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:24.287345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 603/1208] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:04:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:24.374159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:24.501996 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60311f-6939-4a9c-9b55-531d6176b8da None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:24.573214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713d6d2-bebf-4bb5-ab61-ea2911d3d36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:24.574083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713d6d2-bebf-4bb5-ab61-ea2911d3d36e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:24.574301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713d6d2-bebf-4bb5-ab61-ea2911d3d36e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:24.574545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713d6d2-bebf-4bb5-ab61-ea2911d3d36e None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:24.575354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713d6d2-bebf-4bb5-ab61-ea2911d3d36e None admin] RBAC: Adding query filter params (name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:24.575658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713d6d2-bebf-4bb5-ab61-ea2911d3d36e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:24.577318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713d6d2-bebf-4bb5-ab61-ea2911d3d36e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:24.591077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 606/1209] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:04:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:04:24.599292 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8bc74-114e-4c3c-b9d7-bd0f644ca77d None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:24.674148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:24.751147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8bc74-114e-4c3c-b9d7-bd0f644ca77d None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:24.801478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e540e5-170f-4eca-99c9-41c091fb52cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:24.802870 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e540e5-170f-4eca-99c9-41c091fb52cb None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:24.803398 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e540e5-170f-4eca-99c9-41c091fb52cb None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:24.804078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e540e5-170f-4eca-99c9-41c091fb52cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:24.834560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e540e5-170f-4eca-99c9-41c091fb52cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:24.837660 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e540e5-170f-4eca-99c9-41c091fb52cb None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-92e540e5-170f-4eca-99c9-41c091fb52cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:04:24.874406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:24.876602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:04:24.878406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:04:24.878406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:24.878406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:24.878406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 604/1210] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:24.886916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713d6d2-bebf-4bb5-ab61-ea2911d3d36e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:25.101118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:25.110122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713d6d2-bebf-4bb5-ab61-ea2911d3d36e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:25.204843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c39719-6225-4a0f-a659-1bd9fb4ba538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:25.205649 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c39719-6225-4a0f-a659-1bd9fb4ba538 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:25.205926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c39719-6225-4a0f-a659-1bd9fb4ba538 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:25.206257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c39719-6225-4a0f-a659-1bd9fb4ba538 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:25.207034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c39719-6225-4a0f-a659-1bd9fb4ba538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:25.207258 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c39719-6225-4a0f-a659-1bd9fb4ba538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:25.208850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c39719-6225-4a0f-a659-1bd9fb4ba538 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:25.235047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 607/1211] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:25.237868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e540e5-170f-4eca-99c9-41c091fb52cb None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:25.385896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e540e5-170f-4eca-99c9-41c091fb52cb None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:25.388967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:25.439413 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02756700-884a-407d-88cd-95984505c53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:25.440084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02756700-884a-407d-88cd-95984505c53d None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:25.440358 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02756700-884a-407d-88cd-95984505c53d None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:25.440626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02756700-884a-407d-88cd-95984505c53d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:25.454813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02756700-884a-407d-88cd-95984505c53d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:25.456926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02756700-884a-407d-88cd-95984505c53d None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-02756700-884a-407d-88cd-95984505c53d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:04:25.471280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:25.473517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:25.475809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:25.475809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:25.475809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:04:25.475809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:04:25.475809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:04:25.475809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:04:25.475809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:04:25.475809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:04:25.475809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:04:25.475809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:25.475809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 605/1212] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:25.481823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c39719-6225-4a0f-a659-1bd9fb4ba538 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:25.696737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c39719-6225-4a0f-a659-1bd9fb4ba538 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:25.780339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d63407-78bc-4bdf-ac98-ebd4c77df876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:25.781113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d63407-78bc-4bdf-ac98-ebd4c77df876 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:25.781476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d63407-78bc-4bdf-ac98-ebd4c77df876 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:25.781808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d63407-78bc-4bdf-ac98-ebd4c77df876 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:25.782661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d63407-78bc-4bdf-ac98-ebd4c77df876 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:25.782957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d63407-78bc-4bdf-ac98-ebd4c77df876 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:25.784472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d63407-78bc-4bdf-ac98-ebd4c77df876 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:25.812439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 608/1213] 10.222.0.61 () {56 vars in 1143 bytes} [Wed Sep 7 13:04:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:25.820409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02756700-884a-407d-88cd-95984505c53d None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:26.004331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02756700-884a-407d-88cd-95984505c53d None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:26.064378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2810aef-235b-438e-b87e-d2268fbdc39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:26.065049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2810aef-235b-438e-b87e-d2268fbdc39a None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:26.065292 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2810aef-235b-438e-b87e-d2268fbdc39a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:26.065602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2810aef-235b-438e-b87e-d2268fbdc39a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:26.132124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:26.212053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2810aef-235b-438e-b87e-d2268fbdc39a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:26.214956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2810aef-235b-438e-b87e-d2268fbdc39a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:26.409938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 606/1214] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:04:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:04:26.412261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d63407-78bc-4bdf-ac98-ebd4c77df876 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:26.448734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:26.627145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d63407-78bc-4bdf-ac98-ebd4c77df876 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:26.703042 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cebcb-dd66-466b-8222-4a82b531b1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:26.703903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cebcb-dd66-466b-8222-4a82b531b1ad None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:26.704562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cebcb-dd66-466b-8222-4a82b531b1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:26.704861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cebcb-dd66-466b-8222-4a82b531b1ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:26.718721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cebcb-dd66-466b-8222-4a82b531b1ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:26.720760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cebcb-dd66-466b-8222-4a82b531b1ad None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-be2cebcb-dd66-466b-8222-4a82b531b1ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:04:26.739157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:26.747802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:04:26.749170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:04:26.749170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:26.749170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:26.749170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 609/1215] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:26.751502 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2810aef-235b-438e-b87e-d2268fbdc39a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:26.831006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:26.893051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2810aef-235b-438e-b87e-d2268fbdc39a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:26.955634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2627585-6731-4b6e-90fb-c5012afd439b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:26.956395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2627585-6731-4b6e-90fb-c5012afd439b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:26.956788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2627585-6731-4b6e-90fb-c5012afd439b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:26.957262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2627585-6731-4b6e-90fb-c5012afd439b None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:26.958162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2627585-6731-4b6e-90fb-c5012afd439b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:26.958728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2627585-6731-4b6e-90fb-c5012afd439b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:26.961683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2627585-6731-4b6e-90fb-c5012afd439b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:26.985309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 607/1216] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:26.991042 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cebcb-dd66-466b-8222-4a82b531b1ad None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:27.132474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:27.188883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cebcb-dd66-466b-8222-4a82b531b1ad None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:27.260401 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa426f7d-caca-47cc-ae1c-92411f488d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:27.260958 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa426f7d-caca-47cc-ae1c-92411f488d1c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:27.261240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa426f7d-caca-47cc-ae1c-92411f488d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:27.261515 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa426f7d-caca-47cc-ae1c-92411f488d1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:27.269720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa426f7d-caca-47cc-ae1c-92411f488d1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:27.271266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa426f7d-caca-47cc-ae1c-92411f488d1c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-fa426f7d-caca-47cc-ae1c-92411f488d1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:04:27.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:27.294259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:27.296498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:04:27.296498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:04:27.296498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:04:27.296498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:04:27.296498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:04:27.296498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:04:27.296498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:04:27.296498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:27.296498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 610/1217] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:04:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:27.297678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2627585-6731-4b6e-90fb-c5012afd439b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:27.445614 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2627585-6731-4b6e-90fb-c5012afd439b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:27.479259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:27.498461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f49a18-70b4-4466-8e81-358ccad6c009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:27.499350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f49a18-70b4-4466-8e81-358ccad6c009 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:27.499629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f49a18-70b4-4466-8e81-358ccad6c009 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:27.499981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f49a18-70b4-4466-8e81-358ccad6c009 None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:27.500732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f49a18-70b4-4466-8e81-358ccad6c009 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:27.501047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f49a18-70b4-4466-8e81-358ccad6c009 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:27.502571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f49a18-70b4-4466-8e81-358ccad6c009 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:27.525311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 608/1218] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:04:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:27.534514 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa426f7d-caca-47cc-ae1c-92411f488d1c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:27.674445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa426f7d-caca-47cc-ae1c-92411f488d1c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:27.721971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:27.729967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84d9a9-5d1a-471d-8c05-a50acc68d386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:27.730625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84d9a9-5d1a-471d-8c05-a50acc68d386 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:27.730925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84d9a9-5d1a-471d-8c05-a50acc68d386 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:27.731210 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84d9a9-5d1a-471d-8c05-a50acc68d386 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:27.732094 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84d9a9-5d1a-471d-8c05-a50acc68d386 None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=9e283e62741f40439d02e545a212807a) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:27.732424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84d9a9-5d1a-471d-8c05-a50acc68d386 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:27.734191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84d9a9-5d1a-471d-8c05-a50acc68d386 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:27.752064 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 611/1219] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:04:27 2022] GET /identity/v3/role_assignments?user.id=9e283e62741f40439d02e545a212807a&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:30.175905 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce310f80-42a7-4953-82e3-e96186f35865 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:30.176599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce310f80-42a7-4953-82e3-e96186f35865 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:30.176599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce310f80-42a7-4953-82e3-e96186f35865 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:30.178217 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 609/1220] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:30.185644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98771ad8-49b0-4852-8861-f5244b8dfea0 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:30.186468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98771ad8-49b0-4852-8861-f5244b8dfea0 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:30.186468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98771ad8-49b0-4852-8861-f5244b8dfea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:30.211363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-98771ad8-49b0-4852-8861-f5244b8dfea0 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:30.212940 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-98771ad8-49b0-4852-8861-f5244b8dfea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:30.321578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 612/1221] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:04:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:04:30.869593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce310f80-42a7-4953-82e3-e96186f35865 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:31.056541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce310f80-42a7-4953-82e3-e96186f35865 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:31.117736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f4352-482b-4a73-aed4-37c9a4702d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:31.119260 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f4352-482b-4a73-aed4-37c9a4702d12 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:31.119837 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f4352-482b-4a73-aed4-37c9a4702d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:31.120451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f4352-482b-4a73-aed4-37c9a4702d12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:31.140694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f4352-482b-4a73-aed4-37c9a4702d12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-518f4352-482b-4a73-aed4-37c9a4702d12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:04:31.151229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:31.153431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:31.155229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:31.155229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:31.155229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:31.155229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:31.155229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:04:31.155229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:04:31.155229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:04:31.155229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:04:31.155229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:04:31.155229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:04:31.155229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:04:31.155229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:31.155229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 610/1222] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:04:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:31.162812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98771ad8-49b0-4852-8861-f5244b8dfea0 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:31.324987 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98771ad8-49b0-4852-8861-f5244b8dfea0 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:31.360537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:31.383143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a4e0f-ed68-4ed5-9e04-d6f91df85114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:31.383832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a4e0f-ed68-4ed5-9e04-d6f91df85114 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:31.384303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a4e0f-ed68-4ed5-9e04-d6f91df85114 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:31.384702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a4e0f-ed68-4ed5-9e04-d6f91df85114 None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:31.385761 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a4e0f-ed68-4ed5-9e04-d6f91df85114 None admin] RBAC: Adding query filter params (name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:31.386196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a4e0f-ed68-4ed5-9e04-d6f91df85114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:31.387918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a4e0f-ed68-4ed5-9e04-d6f91df85114 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:31.403482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 613/1223] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:04:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:31.409454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f4352-482b-4a73-aed4-37c9a4702d12 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:31.476188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:31.552363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f4352-482b-4a73-aed4-37c9a4702d12 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:31.595570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132a971-181c-408d-9b73-330486c92e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:31.596323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a971-181c-408d-9b73-330486c92e58 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:31.596628 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a971-181c-408d-9b73-330486c92e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:31.597009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a971-181c-408d-9b73-330486c92e58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:31.609819 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132a971-181c-408d-9b73-330486c92e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:31.611189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132a971-181c-408d-9b73-330486c92e58 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-6132a971-181c-408d-9b73-330486c92e58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:04:31.635732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:31.636926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:04:31.638035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:04:31.638035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:31.638035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:31.638035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 611/1224] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:31.648922 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a4e0f-ed68-4ed5-9e04-d6f91df85114 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:31.801338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a4e0f-ed68-4ed5-9e04-d6f91df85114 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:31.860522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34089270-dd66-43c7-b094-3d743b898ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:31.861547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34089270-dd66-43c7-b094-3d743b898ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:31.862030 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34089270-dd66-43c7-b094-3d743b898ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:31.862479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34089270-dd66-43c7-b094-3d743b898ec5 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:31.863762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34089270-dd66-43c7-b094-3d743b898ec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:31.864119 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34089270-dd66-43c7-b094-3d743b898ec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:31.865984 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34089270-dd66-43c7-b094-3d743b898ec5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:31.890147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 614/1225] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:31.891711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132a971-181c-408d-9b73-330486c92e58 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:32.018168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:32.033001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132a971-181c-408d-9b73-330486c92e58 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:32.090741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9edf6-d23c-40d8-97aa-7f563f729635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:32.091433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9edf6-d23c-40d8-97aa-7f563f729635 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:32.091828 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9edf6-d23c-40d8-97aa-7f563f729635 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:32.092165 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9edf6-d23c-40d8-97aa-7f563f729635 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:32.106639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9edf6-d23c-40d8-97aa-7f563f729635 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:32.109305 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9edf6-d23c-40d8-97aa-7f563f729635 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-64f9edf6-d23c-40d8-97aa-7f563f729635 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:04:32.131586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:32.133730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:32.136291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:32.136291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:32.136291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:04:32.136291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:04:32.136291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:04:32.136291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:04:32.136291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:04:32.136291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:04:32.136291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:04:32.136291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:32.137604 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 612/1226] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:32.143591 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34089270-dd66-43c7-b094-3d743b898ec5 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:32.234361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:32.335740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34089270-dd66-43c7-b094-3d743b898ec5 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:32.389831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9548c58-be05-4f18-8c4c-baef4265db02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:32.390762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9548c58-be05-4f18-8c4c-baef4265db02 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:32.391135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9548c58-be05-4f18-8c4c-baef4265db02 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:32.391509 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9548c58-be05-4f18-8c4c-baef4265db02 None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:32.392349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9548c58-be05-4f18-8c4c-baef4265db02 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:32.392654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9548c58-be05-4f18-8c4c-baef4265db02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:32.394265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9548c58-be05-4f18-8c4c-baef4265db02 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:32.422286 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 615/1227] 10.222.0.61 () {56 vars in 1143 bytes} [Wed Sep 7 13:04:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:32.432666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9edf6-d23c-40d8-97aa-7f563f729635 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:32.588760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9edf6-d23c-40d8-97aa-7f563f729635 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:32.629227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f9412-314c-4143-acb9-780d6c40af8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:32.629918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f9412-314c-4143-acb9-780d6c40af8c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:32.630197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f9412-314c-4143-acb9-780d6c40af8c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:32.630476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f9412-314c-4143-acb9-780d6c40af8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:32.644381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f9412-314c-4143-acb9-780d6c40af8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:32.645952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f9412-314c-4143-acb9-780d6c40af8c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-670f9412-314c-4143-acb9-780d6c40af8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:04:32.663845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:32.664998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:04:32.666020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:04:32.666020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:32.666020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:32.666201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 613/1228] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:32.673841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9548c58-be05-4f18-8c4c-baef4265db02 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:32.813054 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9548c58-be05-4f18-8c4c-baef4265db02 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:32.816517 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:32.862058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6497a17-d07b-410b-b183-8ddc2f10cd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:32.862693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6497a17-d07b-410b-b183-8ddc2f10cd34 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:32.862971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6497a17-d07b-410b-b183-8ddc2f10cd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:32.863281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6497a17-d07b-410b-b183-8ddc2f10cd34 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:32.863954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6497a17-d07b-410b-b183-8ddc2f10cd34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:32.864260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6497a17-d07b-410b-b183-8ddc2f10cd34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:32.865863 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6497a17-d07b-410b-b183-8ddc2f10cd34 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:32.882399 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 616/1229] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:32.889084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f9412-314c-4143-acb9-780d6c40af8c None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:32.901659 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:33.031275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f9412-314c-4143-acb9-780d6c40af8c None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:33.078833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fcd82-2698-48cf-8c1d-89ec8a9c1d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:33.079719 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fcd82-2698-48cf-8c1d-89ec8a9c1d96 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:33.080098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fcd82-2698-48cf-8c1d-89ec8a9c1d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:33.080476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fcd82-2698-48cf-8c1d-89ec8a9c1d96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:33.095454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fcd82-2698-48cf-8c1d-89ec8a9c1d96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:33.097529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fcd82-2698-48cf-8c1d-89ec8a9c1d96 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-448fcd82-2698-48cf-8c1d-89ec8a9c1d96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:04:33.113446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:33.114706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:33.116037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:04:33.116037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:04:33.116037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:04:33.116037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:04:33.116037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:04:33.116037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:04:33.116037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:04:33.116037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:33.116037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 614/1230] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:04:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:33.127856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6497a17-d07b-410b-b183-8ddc2f10cd34 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:33.276272 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6497a17-d07b-410b-b183-8ddc2f10cd34 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:33.319998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9571f-05fb-4cf4-9f91-61df009fdef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:33.320702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9571f-05fb-4cf4-9f91-61df009fdef9 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:33.321038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9571f-05fb-4cf4-9f91-61df009fdef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:33.321365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9571f-05fb-4cf4-9f91-61df009fdef9 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:33.322140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9571f-05fb-4cf4-9f91-61df009fdef9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:33.322516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9571f-05fb-4cf4-9f91-61df009fdef9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:33.324153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9571f-05fb-4cf4-9f91-61df009fdef9 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:33.339787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 617/1231] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:04:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:04:33.346348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fcd82-2698-48cf-8c1d-89ec8a9c1d96 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:33.426050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:33.476630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fcd82-2698-48cf-8c1d-89ec8a9c1d96 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:33.521438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77394d46-8eae-4056-a350-5dde5a43a360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:33.522128 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77394d46-8eae-4056-a350-5dde5a43a360 None admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:33.523141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77394d46-8eae-4056-a350-5dde5a43a360 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:33.523575 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77394d46-8eae-4056-a350-5dde5a43a360 None admin] PATH_INFO: `/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/9e283e62741f40439d02e545a212807a/roles/5f8e244e8257481e85e65a504aa4bb74` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:33.543821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77394d46-8eae-4056-a350-5dde5a43a360 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user_id=9e283e62741f40439d02e545a212807a, role_id=5f8e244e8257481e85e65a504aa4bb74)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:33.546420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77394d46-8eae-4056-a350-5dde5a43a360 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:33.570128 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 615/1232] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:04:33 2022] PUT /identity/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/9e283e62741f40439d02e545a212807a/roles/5f8e244e8257481e85e65a504aa4bb74 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:04:36.117142 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5e64f-9959-4def-8f14-ce167cc9cf42 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:36.117725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5e64f-9959-4def-8f14-ce167cc9cf42 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:36.117725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5e64f-9959-4def-8f14-ce167cc9cf42 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:36.119134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 618/1233] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:04:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:36.129236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd956363-c6c9-4aa9-b47e-606355c11c9b None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:36.129846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd956363-c6c9-4aa9-b47e-606355c11c9b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:36.129846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd956363-c6c9-4aa9-b47e-606355c11c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:36.151539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-bd956363-c6c9-4aa9-b47e-606355c11c9b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:36.152783 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bd956363-c6c9-4aa9-b47e-606355c11c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:36.244157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 616/1234] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:04:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:04:36.820873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5e64f-9959-4def-8f14-ce167cc9cf42 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:36.859304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:36.995181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5e64f-9959-4def-8f14-ce167cc9cf42 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:37.051479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34b1c8-2540-4476-8122-0d60b7f9f67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:37.052598 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34b1c8-2540-4476-8122-0d60b7f9f67c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:37.053034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34b1c8-2540-4476-8122-0d60b7f9f67c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:37.054037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34b1c8-2540-4476-8122-0d60b7f9f67c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:37.072059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34b1c8-2540-4476-8122-0d60b7f9f67c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-9b34b1c8-2540-4476-8122-0d60b7f9f67c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:04:37.080212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:37.082193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:37.091801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:37.091801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:37.091801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:37.091801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:37.091801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:04:37.091801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:04:37.091801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:04:37.091801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:04:37.091801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:04:37.091801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:04:37.091801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:04:37.091801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:37.091801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 619/1235] 10.222.0.61 () {56 vars in 1102 bytes} [Wed Sep 7 13:04:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:37.092520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd956363-c6c9-4aa9-b47e-606355c11c9b None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:37.175093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:37.273227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd956363-c6c9-4aa9-b47e-606355c11c9b None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:37.339593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805fa58e-2105-4f18-b20c-448b43ef89e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:37.340244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fa58e-2105-4f18-b20c-448b43ef89e5 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:37.340676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fa58e-2105-4f18-b20c-448b43ef89e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:37.341108 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fa58e-2105-4f18-b20c-448b43ef89e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:37.342237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805fa58e-2105-4f18-b20c-448b43ef89e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:37.342592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805fa58e-2105-4f18-b20c-448b43ef89e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:37.344419 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805fa58e-2105-4f18-b20c-448b43ef89e5 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:37.364661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 617/1236] 10.222.0.61 () {56 vars in 1103 bytes} [Wed Sep 7 13:04:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:37.366946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34b1c8-2540-4476-8122-0d60b7f9f67c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:37.508137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:37.534807 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34b1c8-2540-4476-8122-0d60b7f9f67c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:37.592136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a943e3-748b-45d0-9957-e117c5ee5732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:37.592764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a943e3-748b-45d0-9957-e117c5ee5732 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:37.593047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a943e3-748b-45d0-9957-e117c5ee5732 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:37.593324 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a943e3-748b-45d0-9957-e117c5ee5732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:37.609724 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a943e3-748b-45d0-9957-e117c5ee5732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:37.611485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a943e3-748b-45d0-9957-e117c5ee5732 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-d2a943e3-748b-45d0-9957-e117c5ee5732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:04:37.635684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:37.638371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:04:37.640134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:04:37.640134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:37.640134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:37.640134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 620/1237] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:37.645139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805fa58e-2105-4f18-b20c-448b43ef89e5 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:37.826752 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805fa58e-2105-4f18-b20c-448b43ef89e5 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:37.844082 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:37.882278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af47b9a-ddff-4a90-8af3-a2179a3dea6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:37.883242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af47b9a-ddff-4a90-8af3-a2179a3dea6b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:37.883660 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af47b9a-ddff-4a90-8af3-a2179a3dea6b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:37.884072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af47b9a-ddff-4a90-8af3-a2179a3dea6b None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:37.884856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af47b9a-ddff-4a90-8af3-a2179a3dea6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:37.885253 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af47b9a-ddff-4a90-8af3-a2179a3dea6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:37.887608 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af47b9a-ddff-4a90-8af3-a2179a3dea6b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:37.901581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 618/1238] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:37.907654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a943e3-748b-45d0-9957-e117c5ee5732 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:38.081278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a943e3-748b-45d0-9957-e117c5ee5732 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:38.145444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d270f0a-b7b5-4fd3-b37f-140783e50f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:38.146268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d270f0a-b7b5-4fd3-b37f-140783e50f5b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:38.146707 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d270f0a-b7b5-4fd3-b37f-140783e50f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:38.147192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d270f0a-b7b5-4fd3-b37f-140783e50f5b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:38.151550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:38.159627 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d270f0a-b7b5-4fd3-b37f-140783e50f5b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:38.161819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d270f0a-b7b5-4fd3-b37f-140783e50f5b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-0d270f0a-b7b5-4fd3-b37f-140783e50f5b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:04:38.179487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:38.182249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:38.184920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:38.184920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:38.184920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:04:38.184920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:04:38.184920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:04:38.184920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:04:38.184920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:04:38.184920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:04:38.184920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:04:38.184920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:38.184920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 621/1239] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:38.190360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af47b9a-ddff-4a90-8af3-a2179a3dea6b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:38.353504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af47b9a-ddff-4a90-8af3-a2179a3dea6b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:38.411000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb668972-2cdf-4d4b-ae49-9c3d6aa63157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:38.411677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb668972-2cdf-4d4b-ae49-9c3d6aa63157 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:38.412174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb668972-2cdf-4d4b-ae49-9c3d6aa63157 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:38.412773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb668972-2cdf-4d4b-ae49-9c3d6aa63157 None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:38.414250 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb668972-2cdf-4d4b-ae49-9c3d6aa63157 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:38.414609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb668972-2cdf-4d4b-ae49-9c3d6aa63157 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:38.417324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb668972-2cdf-4d4b-ae49-9c3d6aa63157 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:38.449792 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 619/1240] 10.222.0.61 () {56 vars in 1143 bytes} [Wed Sep 7 13:04:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:38.457193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d270f0a-b7b5-4fd3-b37f-140783e50f5b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:38.590275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d270f0a-b7b5-4fd3-b37f-140783e50f5b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:38.649792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d5d55-c913-4a31-89be-fbf4aa2b0269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:38.650773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d5d55-c913-4a31-89be-fbf4aa2b0269 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:38.651171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d5d55-c913-4a31-89be-fbf4aa2b0269 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:38.651616 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d5d55-c913-4a31-89be-fbf4aa2b0269 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:38.744805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:38.753568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d5d55-c913-4a31-89be-fbf4aa2b0269 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:38.755449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d5d55-c913-4a31-89be-fbf4aa2b0269 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:38.942657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 622/1241] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:04:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 13:04:38.952401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb668972-2cdf-4d4b-ae49-9c3d6aa63157 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:39.002342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:39.130853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb668972-2cdf-4d4b-ae49-9c3d6aa63157 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:39.199985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ebcef2-921f-4723-8ace-82d69ac1f92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:39.201014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebcef2-921f-4723-8ace-82d69ac1f92f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:39.201435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebcef2-921f-4723-8ace-82d69ac1f92f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:39.201920 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebcef2-921f-4723-8ace-82d69ac1f92f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:39.221807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ebcef2-921f-4723-8ace-82d69ac1f92f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:39.224656 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ebcef2-921f-4723-8ace-82d69ac1f92f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-55ebcef2-921f-4723-8ace-82d69ac1f92f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:04:39.256577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:39.258546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:04:39.260577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:04:39.260577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:39.260577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:39.260577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 620/1242] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:39.265158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d5d55-c913-4a31-89be-fbf4aa2b0269 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:39.421563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:39.429214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d5d55-c913-4a31-89be-fbf4aa2b0269 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:39.481447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f849d1f-3fd0-4303-8b9e-3feb7df2d054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:39.482579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f849d1f-3fd0-4303-8b9e-3feb7df2d054 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:39.482579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f849d1f-3fd0-4303-8b9e-3feb7df2d054 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:39.482579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f849d1f-3fd0-4303-8b9e-3feb7df2d054 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:39.483283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f849d1f-3fd0-4303-8b9e-3feb7df2d054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:39.483581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f849d1f-3fd0-4303-8b9e-3feb7df2d054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:39.484988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f849d1f-3fd0-4303-8b9e-3feb7df2d054 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:39.508245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 623/1243] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:04:39.509769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ebcef2-921f-4723-8ace-82d69ac1f92f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:39.628107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:39.673862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ebcef2-921f-4723-8ace-82d69ac1f92f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:39.733088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804425a-79a6-4f48-921b-6e9a65627c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:39.733877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804425a-79a6-4f48-921b-6e9a65627c09 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:39.734228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804425a-79a6-4f48-921b-6e9a65627c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:39.734529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804425a-79a6-4f48-921b-6e9a65627c09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:39.746327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804425a-79a6-4f48-921b-6e9a65627c09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:39.748916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804425a-79a6-4f48-921b-6e9a65627c09 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-a804425a-79a6-4f48-921b-6e9a65627c09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:04:39.769041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:39.771074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:39.773834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:04:39.773834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:04:39.773834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:04:39.773834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:04:39.773834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:04:39.773834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:04:39.773834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:04:39.773834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:39.773834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 621/1244] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:04:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:39.778896 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f849d1f-3fd0-4303-8b9e-3feb7df2d054 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:39.937672 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f849d1f-3fd0-4303-8b9e-3feb7df2d054 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:39.993916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad3ffa-1543-4c31-81b8-3be8144ad172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:39.994808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad3ffa-1543-4c31-81b8-3be8144ad172 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:39.995139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad3ffa-1543-4c31-81b8-3be8144ad172 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:39.995442 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad3ffa-1543-4c31-81b8-3be8144ad172 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:39.996525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad3ffa-1543-4c31-81b8-3be8144ad172 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:39.997048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad3ffa-1543-4c31-81b8-3be8144ad172 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:40.000281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad3ffa-1543-4c31-81b8-3be8144ad172 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:40.023700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 624/1245] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:04:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:40.032153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804425a-79a6-4f48-921b-6e9a65627c09 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:40.191532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804425a-79a6-4f48-921b-6e9a65627c09 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:40.246489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1650f5-011a-436a-84fa-bf2a8f82815f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:40.247914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1650f5-011a-436a-84fa-bf2a8f82815f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:40.247914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1650f5-011a-436a-84fa-bf2a8f82815f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:40.247914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1650f5-011a-436a-84fa-bf2a8f82815f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:40.248670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1650f5-011a-436a-84fa-bf2a8f82815f None admin] RBAC: Adding query filter params (role.id=5f8e244e8257481e85e65a504aa4bb74, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=9e283e62741f40439d02e545a212807a) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:40.248984 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1650f5-011a-436a-84fa-bf2a8f82815f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:40.250714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1650f5-011a-436a-84fa-bf2a8f82815f None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:40.260597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 622/1246] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:04:40 2022] GET /identity/v3/role_assignments?user.id=9e283e62741f40439d02e545a212807a&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=5f8e244e8257481e85e65a504aa4bb74 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:42.952863 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b00d6-fabf-424e-ac20-2f06537bccf5 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:42.953807 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b00d6-fabf-424e-ac20-2f06537bccf5 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:42.953807 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b00d6-fabf-424e-ac20-2f06537bccf5 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:42.955357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 625/1247] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:42.962485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13d371-4d5c-4b98-a408-16e336bc715d None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:42.963530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13d371-4d5c-4b98-a408-16e336bc715d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:42.963530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13d371-4d5c-4b98-a408-16e336bc715d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:42.970955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:42.987385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-5a13d371-4d5c-4b98-a408-16e336bc715d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:42.988747 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5a13d371-4d5c-4b98-a408-16e336bc715d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:43.091782 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 623/1248] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:04:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:04:43.433732 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b00d6-fabf-424e-ac20-2f06537bccf5 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:43.456161 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:43.604789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b00d6-fabf-424e-ac20-2f06537bccf5 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:43.672919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a9868-818d-4895-adf9-0f5513583436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:43.673892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a9868-818d-4895-adf9-0f5513583436 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:43.674310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a9868-818d-4895-adf9-0f5513583436 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:43.674736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a9868-818d-4895-adf9-0f5513583436 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:43.695583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a9868-818d-4895-adf9-0f5513583436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-4f7a9868-818d-4895-adf9-0f5513583436 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:04:43.706215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:43.707805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:43.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:43.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:43.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:43.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:43.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:04:43.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:04:43.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:04:43.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:04:43.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:04:43.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:04:43.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:04:43.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:43.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 626/1249] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:04:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:04:43.717370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13d371-4d5c-4b98-a408-16e336bc715d None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:43.892187 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13d371-4d5c-4b98-a408-16e336bc715d None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:43.938488 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:43.945876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe5678-1970-4cda-b1ff-0b0e9f9d60ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:43.946423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe5678-1970-4cda-b1ff-0b0e9f9d60ec None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:43.946706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe5678-1970-4cda-b1ff-0b0e9f9d60ec None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:43.946986 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe5678-1970-4cda-b1ff-0b0e9f9d60ec None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:43.947726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe5678-1970-4cda-b1ff-0b0e9f9d60ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:43.948048 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe5678-1970-4cda-b1ff-0b0e9f9d60ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:43.949788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe5678-1970-4cda-b1ff-0b0e9f9d60ec None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:43.962261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 624/1250] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:04:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:43.969811 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a9868-818d-4895-adf9-0f5513583436 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:44.119468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:44.153860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a9868-818d-4895-adf9-0f5513583436 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:44.206962 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db8369-0375-40f4-9e09-76be2d51f70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:44.208020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db8369-0375-40f4-9e09-76be2d51f70f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:44.208464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db8369-0375-40f4-9e09-76be2d51f70f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:44.209033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db8369-0375-40f4-9e09-76be2d51f70f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:44.228417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db8369-0375-40f4-9e09-76be2d51f70f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:44.230876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db8369-0375-40f4-9e09-76be2d51f70f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-27db8369-0375-40f4-9e09-76be2d51f70f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:04:44.257496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:44.259619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:04:44.262007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:04:44.262007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:44.262007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:44.262007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 627/1251] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:44.267910 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe5678-1970-4cda-b1ff-0b0e9f9d60ec None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:44.455971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe5678-1970-4cda-b1ff-0b0e9f9d60ec None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:44.527678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a749cf-b98e-45d9-9cb3-1757eaf589d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:44.528869 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a749cf-b98e-45d9-9cb3-1757eaf589d7 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:44.529384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a749cf-b98e-45d9-9cb3-1757eaf589d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:44.530066 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a749cf-b98e-45d9-9cb3-1757eaf589d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:44.531324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a749cf-b98e-45d9-9cb3-1757eaf589d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:44.531863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a749cf-b98e-45d9-9cb3-1757eaf589d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:44.534536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a749cf-b98e-45d9-9cb3-1757eaf589d7 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:44.558366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 625/1252] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:44.566544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db8369-0375-40f4-9e09-76be2d51f70f None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:44.735918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db8369-0375-40f4-9e09-76be2d51f70f None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:44.786062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:44.801343 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4b3c2-8b4d-48ef-839a-3b94b7177406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:44.802391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4b3c2-8b4d-48ef-839a-3b94b7177406 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:44.802843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4b3c2-8b4d-48ef-839a-3b94b7177406 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:44.803386 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4b3c2-8b4d-48ef-839a-3b94b7177406 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:44.814871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4b3c2-8b4d-48ef-839a-3b94b7177406 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:44.816399 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4b3c2-8b4d-48ef-839a-3b94b7177406 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-b3b4b3c2-8b4d-48ef-839a-3b94b7177406 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:04:44.829477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:44.831437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:44.833330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:44.833330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:44.833330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:04:44.833330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:04:44.833330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:04:44.833330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:04:44.833330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:04:44.833330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:04:44.833330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:04:44.833330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:44.833330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 628/1253] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:44.838307 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a749cf-b98e-45d9-9cb3-1757eaf589d7 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:44.940898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:45.024751 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a749cf-b98e-45d9-9cb3-1757eaf589d7 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:45.086756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a0df9-9ade-4a14-adf1-fb408c783c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:45.087442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a0df9-9ade-4a14-adf1-fb408c783c2a None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:45.087725 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a0df9-9ade-4a14-adf1-fb408c783c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:45.088003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a0df9-9ade-4a14-adf1-fb408c783c2a None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:45.088756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a0df9-9ade-4a14-adf1-fb408c783c2a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:45.089062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a0df9-9ade-4a14-adf1-fb408c783c2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:45.090531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a0df9-9ade-4a14-adf1-fb408c783c2a None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:45.117679 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 626/1254] 10.222.0.61 () {56 vars in 1143 bytes} [Wed Sep 7 13:04:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:45.124523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4b3c2-8b4d-48ef-839a-3b94b7177406 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:45.259014 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4b3c2-8b4d-48ef-839a-3b94b7177406 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:45.296693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786ba35-3536-41c7-ae33-124bfbe8e75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:45.297316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786ba35-3536-41c7-ae33-124bfbe8e75b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:45.297591 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786ba35-3536-41c7-ae33-124bfbe8e75b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:45.297857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786ba35-3536-41c7-ae33-124bfbe8e75b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:45.361861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:45.377085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786ba35-3536-41c7-ae33-124bfbe8e75b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:45.378589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786ba35-3536-41c7-ae33-124bfbe8e75b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:45.495347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 629/1255] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:04:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:04:45.501629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a0df9-9ade-4a14-adf1-fb408c783c2a None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:45.615121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a0df9-9ade-4a14-adf1-fb408c783c2a None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:45.632009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:45.655369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fe156-b8fb-4994-91da-eba6cb4448cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:45.656001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fe156-b8fb-4994-91da-eba6cb4448cd None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:45.656281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fe156-b8fb-4994-91da-eba6cb4448cd None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:45.656555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fe156-b8fb-4994-91da-eba6cb4448cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:45.668666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fe156-b8fb-4994-91da-eba6cb4448cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:45.670268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fe156-b8fb-4994-91da-eba6cb4448cd None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-ff1fe156-b8fb-4994-91da-eba6cb4448cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:04:45.687134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:45.688844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:04:45.690220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:04:45.690220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:45.690220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:45.690220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 627/1256] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:45.698083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786ba35-3536-41c7-ae33-124bfbe8e75b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:45.814985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:45.829665 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786ba35-3536-41c7-ae33-124bfbe8e75b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:45.872669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c09ac9-f586-4902-8bed-6e8099653b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:45.873199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c09ac9-f586-4902-8bed-6e8099653b47 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:45.873474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c09ac9-f586-4902-8bed-6e8099653b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:45.873738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c09ac9-f586-4902-8bed-6e8099653b47 None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:45.874392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c09ac9-f586-4902-8bed-6e8099653b47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:45.874690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c09ac9-f586-4902-8bed-6e8099653b47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:45.876220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c09ac9-f586-4902-8bed-6e8099653b47 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:45.888947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 630/1257] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:45.895210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fe156-b8fb-4994-91da-eba6cb4448cd None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:46.020363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fe156-b8fb-4994-91da-eba6cb4448cd None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:46.070952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590825-32c2-4762-ab44-0ed682416bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:46.071637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590825-32c2-4762-ab44-0ed682416bef None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:46.071914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590825-32c2-4762-ab44-0ed682416bef None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:46.072186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590825-32c2-4762-ab44-0ed682416bef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:46.080285 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39590825-32c2-4762-ab44-0ed682416bef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:46.082145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39590825-32c2-4762-ab44-0ed682416bef None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-39590825-32c2-4762-ab44-0ed682416bef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:04:46.097506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:46.099290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:46.100599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:04:46.100599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:04:46.100599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:04:46.100599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:04:46.100599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:04:46.100599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:04:46.100599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:04:46.100599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:46.100599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 628/1258] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:04:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:46.105202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c09ac9-f586-4902-8bed-6e8099653b47 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:46.250802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c09ac9-f586-4902-8bed-6e8099653b47 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:46.291315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b89e5-848c-44e1-b147-75b8c8873448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:46.291940 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b89e5-848c-44e1-b147-75b8c8873448 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:46.292213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b89e5-848c-44e1-b147-75b8c8873448 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:46.292478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b89e5-848c-44e1-b147-75b8c8873448 None admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:46.293181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b89e5-848c-44e1-b147-75b8c8873448 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:46.293484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b89e5-848c-44e1-b147-75b8c8873448 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:46.294942 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b89e5-848c-44e1-b147-75b8c8873448 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:46.309016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 631/1259] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:04:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:46.319372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590825-32c2-4762-ab44-0ed682416bef None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:46.350758 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:46.436617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590825-32c2-4762-ab44-0ed682416bef None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:46.476332 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580af1c0-5e0b-477e-966d-78616b8b0f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:46.476955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580af1c0-5e0b-477e-966d-78616b8b0f84 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:46.477325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580af1c0-5e0b-477e-966d-78616b8b0f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:46.477513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580af1c0-5e0b-477e-966d-78616b8b0f84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:46.478256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580af1c0-5e0b-477e-966d-78616b8b0f84 None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=9e283e62741f40439d02e545a212807a) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:46.478570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580af1c0-5e0b-477e-966d-78616b8b0f84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:46.480276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580af1c0-5e0b-477e-966d-78616b8b0f84 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:46.487740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 629/1260] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:04:46 2022] GET /identity/v3/role_assignments?user.id=9e283e62741f40439d02e545a212807a&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:04:49.709119 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705a1e3-213d-4a9e-b8c9-2ab652561947 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:49.709983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705a1e3-213d-4a9e-b8c9-2ab652561947 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:49.709983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705a1e3-213d-4a9e-b8c9-2ab652561947 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:49.711087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 632/1261] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:04:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:49.718242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8e7c5-5d8a-44e5-b320-47d8b152aaf1 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:49.718831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8e7c5-5d8a-44e5-b320-47d8b152aaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:49.719526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8e7c5-5d8a-44e5-b320-47d8b152aaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:49.752355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-95f8e7c5-5d8a-44e5-b320-47d8b152aaf1 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:49.754781 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-95f8e7c5-5d8a-44e5-b320-47d8b152aaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:49.886680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 630/1262] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:04:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:04:50.166357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705a1e3-213d-4a9e-b8c9-2ab652561947 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:50.178904 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:50.308044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705a1e3-213d-4a9e-b8c9-2ab652561947 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:50.347424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee930b52-76af-4f03-ae31-c97120435738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:50.347985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee930b52-76af-4f03-ae31-c97120435738 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:50.348261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee930b52-76af-4f03-ae31-c97120435738 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:50.348531 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee930b52-76af-4f03-ae31-c97120435738 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:50.361152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee930b52-76af-4f03-ae31-c97120435738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-ee930b52-76af-4f03-ae31-c97120435738 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:04:50.366716 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:50.368464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:50.370192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:50.370192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:50.370192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:50.370192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:50.370192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:04:50.370192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:04:50.370192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:04:50.370192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:04:50.370192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:04:50.370192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:04:50.370192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:04:50.370192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:50.370192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 633/1263] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:04:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:04:50.375987 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8e7c5-5d8a-44e5-b320-47d8b152aaf1 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:50.396662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:50.493736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8e7c5-5d8a-44e5-b320-47d8b152aaf1 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:50.534586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf86399-257a-46a4-b633-c7f5c8633b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:50.535355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf86399-257a-46a4-b633-c7f5c8633b4e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:50.535684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf86399-257a-46a4-b633-c7f5c8633b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:50.536006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf86399-257a-46a4-b633-c7f5c8633b4e None admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:50.536777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf86399-257a-46a4-b633-c7f5c8633b4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:50.537134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf86399-257a-46a4-b633-c7f5c8633b4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:50.538603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf86399-257a-46a4-b633-c7f5c8633b4e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:50.548665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 631/1264] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:04:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:50.555432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee930b52-76af-4f03-ae31-c97120435738 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:50.635194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:50.680862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee930b52-76af-4f03-ae31-c97120435738 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:50.723466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e7746-83c3-4a03-964b-ce2ea3b9b635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:50.724277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e7746-83c3-4a03-964b-ce2ea3b9b635 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:50.725052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e7746-83c3-4a03-964b-ce2ea3b9b635 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:50.725371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e7746-83c3-4a03-964b-ce2ea3b9b635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:50.737314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e7746-83c3-4a03-964b-ce2ea3b9b635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:50.738808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e7746-83c3-4a03-964b-ce2ea3b9b635 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-8b7e7746-83c3-4a03-964b-ce2ea3b9b635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:04:50.755624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:50.757586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:04:50.759784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:04:50.759784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:50.759784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:50.759784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 634/1265] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:04:50.764152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf86399-257a-46a4-b633-c7f5c8633b4e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:50.864096 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:50.889189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf86399-257a-46a4-b633-c7f5c8633b4e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:50.926324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2376c-4438-4500-b5a6-fa0849c5cef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:50.926940 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2376c-4438-4500-b5a6-fa0849c5cef6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:50.927264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2376c-4438-4500-b5a6-fa0849c5cef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:50.927541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2376c-4438-4500-b5a6-fa0849c5cef6 None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:50.928205 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2376c-4438-4500-b5a6-fa0849c5cef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:50.932657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2376c-4438-4500-b5a6-fa0849c5cef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:50.932657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2376c-4438-4500-b5a6-fa0849c5cef6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:50.945268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 632/1266] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:04:50.951520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e7746-83c3-4a03-964b-ce2ea3b9b635 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:51.077154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e7746-83c3-4a03-964b-ce2ea3b9b635 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:51.102428 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:51.123757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e425a3c-28bf-4c44-9375-7cba3454a363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:51.124974 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e425a3c-28bf-4c44-9375-7cba3454a363 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:51.125461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e425a3c-28bf-4c44-9375-7cba3454a363 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:51.125937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e425a3c-28bf-4c44-9375-7cba3454a363 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:51.142336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e425a3c-28bf-4c44-9375-7cba3454a363 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:51.144968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e425a3c-28bf-4c44-9375-7cba3454a363 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-8e425a3c-28bf-4c44-9375-7cba3454a363 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:04:51.167877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:51.169122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:51.170252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:51.170252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:51.170252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:04:51.170252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:04:51.170252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:04:51.170252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:04:51.170252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:04:51.170252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:04:51.170252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:04:51.170252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:51.170252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 635/1267] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:51.176120 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2376c-4438-4500-b5a6-fa0849c5cef6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:51.294325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2376c-4438-4500-b5a6-fa0849c5cef6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:51.334742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2ba66-3432-45d2-9a82-3a550576b26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:51.335404 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ba66-3432-45d2-9a82-3a550576b26b None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:51.335690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ba66-3432-45d2-9a82-3a550576b26b None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:51.335965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ba66-3432-45d2-9a82-3a550576b26b None admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:51.336761 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2ba66-3432-45d2-9a82-3a550576b26b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:51.337068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2ba66-3432-45d2-9a82-3a550576b26b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:51.338532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2ba66-3432-45d2-9a82-3a550576b26b None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:51.350892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:51.362301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 633/1268] 10.222.0.61 () {56 vars in 1143 bytes} [Wed Sep 7 13:04:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:51.368891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e425a3c-28bf-4c44-9375-7cba3454a363 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:51.502199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e425a3c-28bf-4c44-9375-7cba3454a363 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:51.540330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fefa91-19e1-44d5-bf19-52811fd0deb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:51.540985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fefa91-19e1-44d5-bf19-52811fd0deb3 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:51.541264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fefa91-19e1-44d5-bf19-52811fd0deb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:51.541533 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fefa91-19e1-44d5-bf19-52811fd0deb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:51.554408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fefa91-19e1-44d5-bf19-52811fd0deb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:51.556055 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fefa91-19e1-44d5-bf19-52811fd0deb3 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-28fefa91-19e1-44d5-bf19-52811fd0deb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:04:51.573782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:51.575532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:04:51.576798 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:04:51.576798 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:51.576798 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:51.576798 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 636/1269] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:04:51.585100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2ba66-3432-45d2-9a82-3a550576b26b None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:51.770702 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2ba66-3432-45d2-9a82-3a550576b26b None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:51.835427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f749f-7b03-454e-aeec-af829ba1b73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:51.836447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f749f-7b03-454e-aeec-af829ba1b73e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:51.836953 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f749f-7b03-454e-aeec-af829ba1b73e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:51.837426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f749f-7b03-454e-aeec-af829ba1b73e None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:51.838536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f749f-7b03-454e-aeec-af829ba1b73e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:51.839090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f749f-7b03-454e-aeec-af829ba1b73e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:51.841660 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f749f-7b03-454e-aeec-af829ba1b73e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:51.864450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 634/1270] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:51.871373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fefa91-19e1-44d5-bf19-52811fd0deb3 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:51.910391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:51.997123 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fefa91-19e1-44d5-bf19-52811fd0deb3 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:52.038778 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e46ada-0c87-473a-88ba-469cefb18da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:52.039443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e46ada-0c87-473a-88ba-469cefb18da5 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:52.039763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e46ada-0c87-473a-88ba-469cefb18da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:52.040123 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e46ada-0c87-473a-88ba-469cefb18da5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:52.048301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e46ada-0c87-473a-88ba-469cefb18da5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:52.049872 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e46ada-0c87-473a-88ba-469cefb18da5 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-84e46ada-0c87-473a-88ba-469cefb18da5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:04:52.063369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:52.064963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:52.066159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:04:52.066159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:04:52.066159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:04:52.066159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:04:52.066159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:04:52.066159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:04:52.066159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:04:52.066159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:52.066159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 637/1271] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:04:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:04:52.074533 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f749f-7b03-454e-aeec-af829ba1b73e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:52.143869 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:52.190971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f749f-7b03-454e-aeec-af829ba1b73e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:52.230190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615976f-ea80-4c2a-afd5-f222ca1c550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:52.230819 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615976f-ea80-4c2a-afd5-f222ca1c550c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:52.231097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615976f-ea80-4c2a-afd5-f222ca1c550c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:52.231402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615976f-ea80-4c2a-afd5-f222ca1c550c None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:52.232117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615976f-ea80-4c2a-afd5-f222ca1c550c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:52.232418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615976f-ea80-4c2a-afd5-f222ca1c550c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:52.233894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615976f-ea80-4c2a-afd5-f222ca1c550c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:52.248253 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 635/1272] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:04:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:52.254823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e46ada-0c87-473a-88ba-469cefb18da5 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:52.364395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:52.376441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e46ada-0c87-473a-88ba-469cefb18da5 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:52.416515 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e8e8d-3481-4b7b-9698-449d427c0e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:52.417205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e8e8d-3481-4b7b-9698-449d427c0e0e None admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:52.417487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e8e8d-3481-4b7b-9698-449d427c0e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:52.417764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e8e8d-3481-4b7b-9698-449d427c0e0e None admin] PATH_INFO: `/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/9e283e62741f40439d02e545a212807a/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:52.419778 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e8e8d-3481-4b7b-9698-449d427c0e0e None admin] RBAC: Authorizing `identity:create_grant(project_id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user_id=9e283e62741f40439d02e545a212807a, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:52.421256 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e8e8d-3481-4b7b-9698-449d427c0e0e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:52.442607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 638/1273] 10.222.0.61 () {58 vars in 1439 bytes} [Wed Sep 7 13:04:52 2022] PUT /identity/v3/projects/c1ee05b1d00e4f4bbb39b9cecdfe754d/users/9e283e62741f40439d02e545a212807a/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:04:54.746633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc922f-e447-4042-8c27-2e24cb9f240d None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:54.747295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc922f-e447-4042-8c27-2e24cb9f240d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:54.747295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc922f-e447-4042-8c27-2e24cb9f240d None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:54.748757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 636/1274] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:04:54.754818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83932a6-17cf-48c7-b345-1c679c93e98d None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:54.755467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83932a6-17cf-48c7-b345-1c679c93e98d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:54.755467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83932a6-17cf-48c7-b345-1c679c93e98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:54.775029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f83932a6-17cf-48c7-b345-1c679c93e98d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:04:54.776310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f83932a6-17cf-48c7-b345-1c679c93e98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:04:54.859465 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 639/1275] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:04:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:04:55.137144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc922f-e447-4042-8c27-2e24cb9f240d None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:55.281565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc922f-e447-4042-8c27-2e24cb9f240d None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:55.304751 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:55.327627 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46d304-2e84-46ff-ad58-eef70d3ba58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:55.328185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46d304-2e84-46ff-ad58-eef70d3ba58f None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:55.328470 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46d304-2e84-46ff-ad58-eef70d3ba58f None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:55.328744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46d304-2e84-46ff-ad58-eef70d3ba58f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:55.341009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46d304-2e84-46ff-ad58-eef70d3ba58f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-8c46d304-2e84-46ff-ad58-eef70d3ba58f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:04:55.346183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:55.347943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:55.349639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:55.349639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:55.349639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:55.349639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:55.349639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:04:55.349639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:04:55.349639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:04:55.349639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:04:55.349639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:04:55.349639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:04:55.349639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:04:55.349639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:55.349639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 637/1276] 10.222.0.61 () {56 vars in 1096 bytes} [Wed Sep 7 13:04:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:04:55.355135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83932a6-17cf-48c7-b345-1c679c93e98d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:55.468979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:55.493597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83932a6-17cf-48c7-b345-1c679c93e98d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:55.534342 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33422df1-0b83-4036-915a-8078c9ea6d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:55.534972 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33422df1-0b83-4036-915a-8078c9ea6d2b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:55.535288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33422df1-0b83-4036-915a-8078c9ea6d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:55.535597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33422df1-0b83-4036-915a-8078c9ea6d2b None admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:55.536319 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33422df1-0b83-4036-915a-8078c9ea6d2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:55.536654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33422df1-0b83-4036-915a-8078c9ea6d2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:55.538255 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33422df1-0b83-4036-915a-8078c9ea6d2b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:55.547752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 640/1277] 10.222.0.61 () {56 vars in 1099 bytes} [Wed Sep 7 13:04:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:55.557220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46d304-2e84-46ff-ad58-eef70d3ba58f None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:55.689215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46d304-2e84-46ff-ad58-eef70d3ba58f None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:55.730457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5eebc4-e471-4962-962c-6eef97a29651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:55.731274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5eebc4-e471-4962-962c-6eef97a29651 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:55.731522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5eebc4-e471-4962-962c-6eef97a29651 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:55.731699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5eebc4-e471-4962-962c-6eef97a29651 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:55.744769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5eebc4-e471-4962-962c-6eef97a29651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:55.746255 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5eebc4-e471-4962-962c-6eef97a29651 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-5f5eebc4-e471-4962-962c-6eef97a29651 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:04:55.763225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:55.765434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:04:55.767274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:04:55.767274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:55.767274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:55.769281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 638/1278] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:55.770456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33422df1-0b83-4036-915a-8078c9ea6d2b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:55.947059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33422df1-0b83-4036-915a-8078c9ea6d2b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:55.989038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5b0f1-74f7-4b81-b137-42be1ea40c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:55.990445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5b0f1-74f7-4b81-b137-42be1ea40c6b None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:55.990725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5b0f1-74f7-4b81-b137-42be1ea40c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:55.990996 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5b0f1-74f7-4b81-b137-42be1ea40c6b None admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:55.991699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5b0f1-74f7-4b81-b137-42be1ea40c6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:55.991999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5b0f1-74f7-4b81-b137-42be1ea40c6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:55.993437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5b0f1-74f7-4b81-b137-42be1ea40c6b None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:56.009447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 641/1279] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:04:56.015776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5eebc4-e471-4962-962c-6eef97a29651 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:56.051742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:56.137898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5eebc4-e471-4962-962c-6eef97a29651 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:56.178429 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fd801-97ba-4cf3-a5d6-abf138f0a1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:56.179310 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fd801-97ba-4cf3-a5d6-abf138f0a1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:56.179595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fd801-97ba-4cf3-a5d6-abf138f0a1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:56.179872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fd801-97ba-4cf3-a5d6-abf138f0a1b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:56.189578 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fd801-97ba-4cf3-a5d6-abf138f0a1b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:56.191611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fd801-97ba-4cf3-a5d6-abf138f0a1b6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-f29fd801-97ba-4cf3-a5d6-abf138f0a1b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:04:56.211893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:56.213841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:56.215789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:56.215789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:56.215789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:04:56.215789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:04:56.215789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:04:56.215789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:04:56.215789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:04:56.215789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:04:56.215789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:04:56.215789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:04:56.215789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 639/1280] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:56.221307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5b0f1-74f7-4b81-b137-42be1ea40c6b None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:56.226449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:56.378776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5b0f1-74f7-4b81-b137-42be1ea40c6b None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:56.419695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead98888-a99b-4323-8565-3ea35d39d45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:56.420322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead98888-a99b-4323-8565-3ea35d39d45e None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:56.420604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead98888-a99b-4323-8565-3ea35d39d45e None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:56.420873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead98888-a99b-4323-8565-3ea35d39d45e None admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:56.421625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead98888-a99b-4323-8565-3ea35d39d45e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:56.421932 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead98888-a99b-4323-8565-3ea35d39d45e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:56.423438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead98888-a99b-4323-8565-3ea35d39d45e None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:56.444613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 642/1281] 10.222.0.61 () {56 vars in 1143 bytes} [Wed Sep 7 13:04:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:56.450837 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fd801-97ba-4cf3-a5d6-abf138f0a1b6 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:56.556760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:56.568648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fd801-97ba-4cf3-a5d6-abf138f0a1b6 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:56.606455 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65983782-072b-45bf-b5d5-fd8272003166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:56.607240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65983782-072b-45bf-b5d5-fd8272003166 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:56.607581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65983782-072b-45bf-b5d5-fd8272003166 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:56.607899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65983782-072b-45bf-b5d5-fd8272003166 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:56.686831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65983782-072b-45bf-b5d5-fd8272003166 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:56.688375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65983782-072b-45bf-b5d5-fd8272003166 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:56.794701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:56.807185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 640/1282] 10.222.0.61 () {58 vars in 1282 bytes} [Wed Sep 7 13:04:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 13:04:56.815468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead98888-a99b-4323-8565-3ea35d39d45e None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:56.919725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:56.990791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead98888-a99b-4323-8565-3ea35d39d45e None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:57.035687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0e6b2-f9fb-4251-8e3e-7074ef4d9757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:57.036368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0e6b2-f9fb-4251-8e3e-7074ef4d9757 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:57.036646 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0e6b2-f9fb-4251-8e3e-7074ef4d9757 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:57.036917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0e6b2-f9fb-4251-8e3e-7074ef4d9757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:57.049985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0e6b2-f9fb-4251-8e3e-7074ef4d9757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:57.051523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0e6b2-f9fb-4251-8e3e-7074ef4d9757 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-69f0e6b2-f9fb-4251-8e3e-7074ef4d9757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:04:57.070971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:57.072934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:04:57.074754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:04:57.074754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:04:57.074754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:57.074754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 643/1283] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:04:57.081114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65983782-072b-45bf-b5d5-fd8272003166 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:57.209025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65983782-072b-45bf-b5d5-fd8272003166 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:57.254990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07384e-29a4-48b7-882e-4c9a20e2265c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:57.256187 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07384e-29a4-48b7-882e-4c9a20e2265c None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:57.256468 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07384e-29a4-48b7-882e-4c9a20e2265c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:57.256748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07384e-29a4-48b7-882e-4c9a20e2265c None admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:57.257422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07384e-29a4-48b7-882e-4c9a20e2265c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:57.257727 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07384e-29a4-48b7-882e-4c9a20e2265c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:57.259166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07384e-29a4-48b7-882e-4c9a20e2265c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:57.273225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 641/1284] 10.222.0.61 () {56 vars in 1109 bytes} [Wed Sep 7 13:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:57.279806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0e6b2-f9fb-4251-8e3e-7074ef4d9757 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:57.469178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0e6b2-f9fb-4251-8e3e-7074ef4d9757 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:57.496533 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:57.521589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01601e97-ddd1-442d-885f-cc021361db7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:57.522264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01601e97-ddd1-442d-885f-cc021361db7c None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:57.522547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01601e97-ddd1-442d-885f-cc021361db7c None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:57.522821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01601e97-ddd1-442d-885f-cc021361db7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:57.530889 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01601e97-ddd1-442d-885f-cc021361db7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:57.532436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01601e97-ddd1-442d-885f-cc021361db7c None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application [None req-01601e97-ddd1-442d-885f-cc021361db7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:04:57.546454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application with Lock( Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:04:57.548131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:04:57.549809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:04:57.549809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:04:57.549809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:04:57.549809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:04:57.549809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:04:57.549809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:04:57.549809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:04:57.549809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: ERROR keystone.server.flask.application Sep 07 13:04:57.551308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 644/1285] 10.222.0.61 () {56 vars in 1111 bytes} [Wed Sep 7 13:04:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:04:57.554263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07384e-29a4-48b7-882e-4c9a20e2265c None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:57.612476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:04:57.668867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07384e-29a4-48b7-882e-4c9a20e2265c None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:57.708207 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc89d1f-2085-4b34-8784-57afc53a26cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:57.709264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc89d1f-2085-4b34-8784-57afc53a26cf None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:57.709767 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc89d1f-2085-4b34-8784-57afc53a26cf None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:57.710261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc89d1f-2085-4b34-8784-57afc53a26cf None admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:57.711582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc89d1f-2085-4b34-8784-57afc53a26cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:57.712124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc89d1f-2085-4b34-8784-57afc53a26cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:57.714848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc89d1f-2085-4b34-8784-57afc53a26cf None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:57.740703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 642/1286] 10.222.0.61 () {56 vars in 1148 bytes} [Wed Sep 7 13:04:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:04:57.747781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01601e97-ddd1-442d-885f-cc021361db7c None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:04:57.924239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01601e97-ddd1-442d-885f-cc021361db7c None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:04:57.963689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e89804-1050-4121-a44e-a0f5b176f822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:04:57.964304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e89804-1050-4121-a44e-a0f5b176f822 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:04:57.964579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e89804-1050-4121-a44e-a0f5b176f822 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:04:57.964845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e89804-1050-4121-a44e-a0f5b176f822 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:04:57.965593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e89804-1050-4121-a44e-a0f5b176f822 None admin] RBAC: Adding query filter params (role.id=40c2d707402d45a0b996aecfd1f11a2f, scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d, user.id=9e283e62741f40439d02e545a212807a) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:04:57.966148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e89804-1050-4121-a44e-a0f5b176f822 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:04:57.968100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e89804-1050-4121-a44e-a0f5b176f822 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:04:57.975572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 645/1287] 10.222.0.61 () {56 vars in 1374 bytes} [Wed Sep 7 13:04:57 2022] GET /identity/v3/role_assignments?user.id=9e283e62741f40439d02e545a212807a&scope.project.id=c1ee05b1d00e4f4bbb39b9cecdfe754d&role.id=40c2d707402d45a0b996aecfd1f11a2f => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:05:00.417610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d716232-e2e5-4876-b7b6-53a8c19d58fd None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:00.418163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d716232-e2e5-4876-b7b6-53a8c19d58fd None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:00.418288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d716232-e2e5-4876-b7b6-53a8c19d58fd None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:00.419874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 643/1288] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:05:00.427991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64722568-d738-4def-9711-488584111b99 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:00.428557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64722568-d738-4def-9711-488584111b99 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:00.428557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64722568-d738-4def-9711-488584111b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:00.449514 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-64722568-d738-4def-9711-488584111b99 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:05:00.450772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-64722568-d738-4def-9711-488584111b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:05:00.486956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:05:00.530816 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 646/1289] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:05:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:05:00.804067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d716232-e2e5-4876-b7b6-53a8c19d58fd None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:05:00.927834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d716232-e2e5-4876-b7b6-53a8c19d58fd None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:05:00.941139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:05:00.968982 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95e65c-5bf3-48d5-8872-86ae5ccf9cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:05:00.969696 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95e65c-5bf3-48d5-8872-86ae5ccf9cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:00.970027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95e65c-5bf3-48d5-8872-86ae5ccf9cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:00.970353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95e65c-5bf3-48d5-8872-86ae5ccf9cb8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:00.971010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95e65c-5bf3-48d5-8872-86ae5ccf9cb8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:05:00.972509 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95e65c-5bf3-48d5-8872-86ae5ccf9cb8 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-0c95e65c-5bf3-48d5-8872-86ae5ccf9cb8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:05:00.982705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:05:00.984665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:05:00.986244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 644/1290] 10.222.0.61 () {56 vars in 1117 bytes} [Wed Sep 7 13:05:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:05:00.994413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64722568-d738-4def-9711-488584111b99 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:05:01.108882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64722568-d738-4def-9711-488584111b99 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:05:01.150615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5fcf4-3f47-4a3b-945a-b16ac61295f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:05:01.151282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5fcf4-3f47-4a3b-945a-b16ac61295f4 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:01.151562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5fcf4-3f47-4a3b-945a-b16ac61295f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:01.151836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5fcf4-3f47-4a3b-945a-b16ac61295f4 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:01.152520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5fcf4-3f47-4a3b-945a-b16ac61295f4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:05:01.152826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5fcf4-3f47-4a3b-945a-b16ac61295f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:05:01.154301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5fcf4-3f47-4a3b-945a-b16ac61295f4 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:05:01.160497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 647/1291] 10.222.0.61 () {56 vars in 1116 bytes} [Wed Sep 7 13:05:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:05:01.170306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95e65c-5bf3-48d5-8872-86ae5ccf9cb8 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:05:01.313310 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95e65c-5bf3-48d5-8872-86ae5ccf9cb8 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:05:01.352743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a54f9-e543-429e-8684-1be2519390d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:05:01.353355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a54f9-e543-429e-8684-1be2519390d2 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:01.353634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a54f9-e543-429e-8684-1be2519390d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:01.353904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a54f9-e543-429e-8684-1be2519390d2 None admin] PATH_INFO: `/v3/services` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:01.354557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a54f9-e543-429e-8684-1be2519390d2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:05:01.354863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a54f9-e543-429e-8684-1be2519390d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:05:01.356323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a54f9-e543-429e-8684-1be2519390d2 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:05:01.362395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 645/1292] 10.222.0.61 () {56 vars in 1116 bytes} [Wed Sep 7 13:05:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:05:03.704445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031060df-0f97-4386-9dd6-bebd309ca620 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:03.704990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031060df-0f97-4386-9dd6-bebd309ca620 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:03.704990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031060df-0f97-4386-9dd6-bebd309ca620 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:03.706322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 648/1293] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:05:03.713383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9934e8-5ef1-4713-83f3-0aa77077f9fd None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:03.713928 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9934e8-5ef1-4713-83f3-0aa77077f9fd None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:03.713928 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9934e8-5ef1-4713-83f3-0aa77077f9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:03.733749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-ad9934e8-5ef1-4713-83f3-0aa77077f9fd None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:05:03.734972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ad9934e8-5ef1-4713-83f3-0aa77077f9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:05:03.771845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:05:03.814972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 646/1294] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:05:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:05:04.092002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031060df-0f97-4386-9dd6-bebd309ca620 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:05:04.128230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:05:04.212071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031060df-0f97-4386-9dd6-bebd309ca620 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:05:04.251786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1317cb-6380-4511-806f-8a87884fe3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:05:04.252415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1317cb-6380-4511-806f-8a87884fe3d0 None admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:04.252691 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1317cb-6380-4511-806f-8a87884fe3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:04.252959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1317cb-6380-4511-806f-8a87884fe3d0 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:04.253645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1317cb-6380-4511-806f-8a87884fe3d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:05:04.255139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1317cb-6380-4511-806f-8a87884fe3d0 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:05:04.272508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 649/1295] 10.222.0.61 () {60 vars in 1141 bytes} [Wed Sep 7 13:05:04 2022] POST /identity/v3/services => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:05:06.661927 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e5480-428a-42e7-9c21-8acce9cf6bce None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:06.662516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e5480-428a-42e7-9c21-8acce9cf6bce None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:06.662516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e5480-428a-42e7-9c21-8acce9cf6bce None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:06.664133 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 647/1296] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:05:06.670667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3ad02-e6e7-40c9-85a9-bc4a2ed39ceb None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:06.671272 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3ad02-e6e7-40c9-85a9-bc4a2ed39ceb None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:06.671425 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3ad02-e6e7-40c9-85a9-bc4a2ed39ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:06.696381 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-70b3ad02-e6e7-40c9-85a9-bc4a2ed39ceb None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:05:06.697693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-70b3ad02-e6e7-40c9-85a9-bc4a2ed39ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:05:06.787542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 650/1297] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:05:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:05:07.067986 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e5480-428a-42e7-9c21-8acce9cf6bce None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:05:07.187570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e5480-428a-42e7-9c21-8acce9cf6bce None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:05:07.226374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a96f1f-3507-413c-9612-64a0dc025f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:05:07.227126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a96f1f-3507-413c-9612-64a0dc025f0e None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:07.227409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a96f1f-3507-413c-9612-64a0dc025f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:07.227688 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a96f1f-3507-413c-9612-64a0dc025f0e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:07.228288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a96f1f-3507-413c-9612-64a0dc025f0e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:05:07.229720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a96f1f-3507-413c-9612-64a0dc025f0e None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-c1a96f1f-3507-413c-9612-64a0dc025f0e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:05:07.240011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:05:07.242390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:05:07.244236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:05:07.247054 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 648/1298] 10.222.0.61 () {56 vars in 1117 bytes} [Wed Sep 7 13:05:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:05:07.249275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3ad02-e6e7-40c9-85a9-bc4a2ed39ceb None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:05:07.284352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:05:07.425137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3ad02-e6e7-40c9-85a9-bc4a2ed39ceb None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:05:07.466993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ced445-19d5-4ac0-a62a-c2ef988fe062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:05:07.467762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ced445-19d5-4ac0-a62a-c2ef988fe062 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:07.467920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ced445-19d5-4ac0-a62a-c2ef988fe062 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:07.468174 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ced445-19d5-4ac0-a62a-c2ef988fe062 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:07.468831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ced445-19d5-4ac0-a62a-c2ef988fe062 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:05:07.469132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ced445-19d5-4ac0-a62a-c2ef988fe062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:05:07.471017 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ced445-19d5-4ac0-a62a-c2ef988fe062 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:05:07.477378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 651/1299] 10.222.0.61 () {56 vars in 1116 bytes} [Wed Sep 7 13:05:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:05:07.484175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a96f1f-3507-413c-9612-64a0dc025f0e None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:05:07.523735 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:05:07.647221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a96f1f-3507-413c-9612-64a0dc025f0e None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:05:07.689957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629e022-cada-4433-9694-e7020a7b5ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:05:07.690646 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629e022-cada-4433-9694-e7020a7b5ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:07.690791 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629e022-cada-4433-9694-e7020a7b5ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:07.691010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629e022-cada-4433-9694-e7020a7b5ea6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:07.691790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629e022-cada-4433-9694-e7020a7b5ea6 None admin] RBAC: Adding query filter params (interface=public, service_id=4aad518dcfa14ca09e7633e29455f020, region_id=RegionOne) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:05:07.692100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629e022-cada-4433-9694-e7020a7b5ea6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:05:07.694073 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629e022-cada-4433-9694-e7020a7b5ea6 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:05:07.700441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 649/1300] 10.222.0.61 () {56 vars in 1251 bytes} [Wed Sep 7 13:05:07 2022] GET /identity/v3/endpoints?service_id=4aad518dcfa14ca09e7633e29455f020&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:05:07.706707 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ced445-19d5-4ac0-a62a-c2ef988fe062 None admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:05:07.848082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:05:07.872585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ced445-19d5-4ac0-a62a-c2ef988fe062 None admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:05:07.927374 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088af14-e41e-476a-abe5-d8df2ecb0011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:05:07.928432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088af14-e41e-476a-abe5-d8df2ecb0011 None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:07.928922 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088af14-e41e-476a-abe5-d8df2ecb0011 None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:07.929379 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088af14-e41e-476a-abe5-d8df2ecb0011 None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:07.930515 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088af14-e41e-476a-abe5-d8df2ecb0011 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:05:07.933144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088af14-e41e-476a-abe5-d8df2ecb0011 None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:05:07.947312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 652/1301] 10.222.0.61 () {56 vars in 1087 bytes} [Wed Sep 7 13:05:07 2022] GET /identity/v3/services => generated 2191 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:05:10.260461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025ca9e-b512-4f1d-8556-5a9f3713b951 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:10.261062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025ca9e-b512-4f1d-8556-5a9f3713b951 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:10.261062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025ca9e-b512-4f1d-8556-5a9f3713b951 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:10.262422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 650/1302] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:05:10.268766 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde88745-41e6-46e2-a808-574fda59518d None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:10.269331 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde88745-41e6-46e2-a808-574fda59518d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:10.269476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde88745-41e6-46e2-a808-574fda59518d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:10.289461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-bde88745-41e6-46e2-a808-574fda59518d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:05:10.290706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-bde88745-41e6-46e2-a808-574fda59518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:05:10.372011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 653/1303] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:05:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:05:10.647406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025ca9e-b512-4f1d-8556-5a9f3713b951 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:05:10.756826 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:05:10.764497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025ca9e-b512-4f1d-8556-5a9f3713b951 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:05:10.803027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e3c1e-ada8-45a0-89d2-d5d268b579b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:05:10.803760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e3c1e-ada8-45a0-89d2-d5d268b579b2 None admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:10.804088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e3c1e-ada8-45a0-89d2-d5d268b579b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:10.804407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e3c1e-ada8-45a0-89d2-d5d268b579b2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:10.805094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e3c1e-ada8-45a0-89d2-d5d268b579b2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:05:10.806642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e3c1e-ada8-45a0-89d2-d5d268b579b2 None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application [None req-1e3e3c1e-ada8-45a0-89d2-d5d268b579b2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:05:10.816810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:05:10.818644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application with Lock( Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: ERROR keystone.server.flask.application Sep 07 13:05:10.819803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 651/1304] 10.222.0.61 () {56 vars in 1117 bytes} [Wed Sep 7 13:05:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:05:10.824988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde88745-41e6-46e2-a808-574fda59518d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:05:10.912460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:05:10.939030 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde88745-41e6-46e2-a808-574fda59518d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:05:10.981912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107c5f0-31c3-4f1b-bb25-289b9dea787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:05:10.982613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107c5f0-31c3-4f1b-bb25-289b9dea787f None admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:10.982936 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107c5f0-31c3-4f1b-bb25-289b9dea787f None admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:10.983283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107c5f0-31c3-4f1b-bb25-289b9dea787f None admin] PATH_INFO: `/v3/services` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:10.983976 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107c5f0-31c3-4f1b-bb25-289b9dea787f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:05:10.984326 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107c5f0-31c3-4f1b-bb25-289b9dea787f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:05:10.985791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107c5f0-31c3-4f1b-bb25-289b9dea787f None admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:05:10.992828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 654/1305] 10.222.0.61 () {56 vars in 1116 bytes} [Wed Sep 7 13:05:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:05:11.000232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e3c1e-ada8-45a0-89d2-d5d268b579b2 None admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:05:11.120278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e3c1e-ada8-45a0-89d2-d5d268b579b2 None admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:05:11.160881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1fffa0-0b79-46ba-abcd-0bd839dafd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:05:11.162148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fffa0-0b79-46ba-abcd-0bd839dafd8c None admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:05:11.162421 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fffa0-0b79-46ba-abcd-0bd839dafd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:05:11.162881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1fffa0-0b79-46ba-abcd-0bd839dafd8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:05:11.163791 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fffa0-0b79-46ba-abcd-0bd839dafd8c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:05:11.165200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1fffa0-0b79-46ba-abcd-0bd839dafd8c None admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:05:11.188505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 652/1306] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:05:10 2022] POST /identity/v3/endpoints => generated 331 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:05:20.989313 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=30 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:05:21.183606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:27.620034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f078ec-9870-47a3-b78a-220d360a7262 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:27.621297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f078ec-9870-47a3-b78a-220d360a7262 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:27.622123 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f078ec-9870-47a3-b78a-220d360a7262 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:27.625704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 655/1307] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:06:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:27.631882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac4262-80fa-4869-96e3-279abddee994 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:27.632790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac4262-80fa-4869-96e3-279abddee994 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:27.633413 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac4262-80fa-4869-96e3-279abddee994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:27.741541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-57ac4262-80fa-4869-96e3-279abddee994 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:27.742819 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-57ac4262-80fa-4869-96e3-279abddee994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:27.884070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 653/1308] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:06:28.197397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecdd0f-b5b1-4992-bbf4-5fd340b0b58c None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:28.197955 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecdd0f-b5b1-4992-bbf4-5fd340b0b58c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:28.198393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecdd0f-b5b1-4992-bbf4-5fd340b0b58c None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:28.200317 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 656/1309] 10.222.0.61 () {54 vars in 965 bytes} [Wed Sep 7 13:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:28.208264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da8e26-137a-42af-a1d4-faabd94a6add None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:28.208560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da8e26-137a-42af-a1d4-faabd94a6add None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:28.208844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da8e26-137a-42af-a1d4-faabd94a6add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:28.408849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-77da8e26-137a-42af-a1d4-faabd94a6add None None] MFA Rules not processed for user `de05c08e30dd4fff93f1c97aac9553d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:28.413349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-77da8e26-137a-42af-a1d4-faabd94a6add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:28.483078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:28.631294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 654/1310] 10.222.0.61 () {58 vars in 1062 bytes} [Wed Sep 7 13:06:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:28.641550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecdd0f-b5b1-4992-bbf4-5fd340b0b58c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:28.643455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-83ecdd0f-b5b1-4992-bbf4-5fd340b0b58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:28.888568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecdd0f-b5b1-4992-bbf4-5fd340b0b58c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:29.070854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0ca05-c598-44eb-91b0-73204e9b8ffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:29.077636 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0ca05-c598-44eb-91b0-73204e9b8ffa service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:29.078359 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0ca05-c598-44eb-91b0-73204e9b8ffa service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:29.079129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0ca05-c598-44eb-91b0-73204e9b8ffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:29.082664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0ae0ca05-c598-44eb-91b0-73204e9b8ffa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:29.219155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:29.354112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0ca05-c598-44eb-91b0-73204e9b8ffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:29.357991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0ca05-c598-44eb-91b0-73204e9b8ffa service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:29.404155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 657/1311] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 763 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:32.277357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027ad63-dd94-4121-81b9-abc9fd5e8db5 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:32.278924 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027ad63-dd94-4121-81b9-abc9fd5e8db5 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:32.279198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027ad63-dd94-4121-81b9-abc9fd5e8db5 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:32.281389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 655/1312] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:06:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:32.290339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34753fd7-913d-486f-967c-5b8f17aa5f9b None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:32.290934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34753fd7-913d-486f-967c-5b8f17aa5f9b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:32.291090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34753fd7-913d-486f-967c-5b8f17aa5f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:32.416043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-34753fd7-913d-486f-967c-5b8f17aa5f9b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:32.418191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-34753fd7-913d-486f-967c-5b8f17aa5f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:32.617149 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 658/1313] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:32.916424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027ad63-dd94-4121-81b9-abc9fd5e8db5 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:32.918066 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d027ad63-dd94-4121-81b9-abc9fd5e8db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:33.105449 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027ad63-dd94-4121-81b9-abc9fd5e8db5 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:33.229484 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:33.233842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc00179-cd1d-495d-b184-26a55c3de802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:33.234680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc00179-cd1d-495d-b184-26a55c3de802 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:33.235266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc00179-cd1d-495d-b184-26a55c3de802 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:33.235863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc00179-cd1d-495d-b184-26a55c3de802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:33.237728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9cc00179-cd1d-495d-b184-26a55c3de802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:33.393342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc00179-cd1d-495d-b184-26a55c3de802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:33.394827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc00179-cd1d-495d-b184-26a55c3de802 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:33.419656 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 656/1314] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:06:36.586511 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc516a9-e947-49bb-b747-21c261f9fa9b None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:36.587357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc516a9-e947-49bb-b747-21c261f9fa9b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:36.587357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc516a9-e947-49bb-b747-21c261f9fa9b None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:36.588743 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 659/1315] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:06:36.597805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea0256-66f0-4a52-9e5c-7e67aa09cec9 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:36.598669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea0256-66f0-4a52-9e5c-7e67aa09cec9 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:36.598832 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea0256-66f0-4a52-9e5c-7e67aa09cec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:36.683050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-96ea0256-66f0-4a52-9e5c-7e67aa09cec9 None None] MFA Rules not processed for user `d7b269b260eb4a96aa4bc25ce7962e27`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:36.684593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-96ea0256-66f0-4a52-9e5c-7e67aa09cec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:36.745662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:36.799774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 657/1316] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:06:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:37.093516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc516a9-e947-49bb-b747-21c261f9fa9b None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:37.095133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-9cc516a9-e947-49bb-b747-21c261f9fa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:37.106070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:37.283929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc516a9-e947-49bb-b747-21c261f9fa9b None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:37.422049 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be5e29-11dd-4593-8953-228a53fae1f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:37.422983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be5e29-11dd-4593-8953-228a53fae1f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:37.423472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be5e29-11dd-4593-8953-228a53fae1f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:37.423917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be5e29-11dd-4593-8953-228a53fae1f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:37.426947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-18be5e29-11dd-4593-8953-228a53fae1f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:37.546548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:37.629550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be5e29-11dd-4593-8953-228a53fae1f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:37.632178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be5e29-11dd-4593-8953-228a53fae1f0 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:37.656424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 660/1317] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:40.597419 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d12a7-b270-4088-880b-b32768493501 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:40.598050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d12a7-b270-4088-880b-b32768493501 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:40.598196 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d12a7-b270-4088-880b-b32768493501 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:40.600315 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 658/1318] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:06:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:40.606790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfeb60-9e21-4a95-8b2a-10dfd44ddd98 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:40.607623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfeb60-9e21-4a95-8b2a-10dfd44ddd98 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:40.607976 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfeb60-9e21-4a95-8b2a-10dfd44ddd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:40.686963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-6cbfeb60-9e21-4a95-8b2a-10dfd44ddd98 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:40.688780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6cbfeb60-9e21-4a95-8b2a-10dfd44ddd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:40.806410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 661/1319] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:06:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:41.114306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890b138-b06b-4c5f-913e-d0df17e10c37 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:41.114610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890b138-b06b-4c5f-913e-d0df17e10c37 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:41.114858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890b138-b06b-4c5f-913e-d0df17e10c37 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:41.116910 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 659/1320] 10.222.0.61 () {54 vars in 965 bytes} [Wed Sep 7 13:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:06:41.124268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930556e-9f01-4e12-8dbf-549a7cca017c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:41.124612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930556e-9f01-4e12-8dbf-549a7cca017c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:41.124924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930556e-9f01-4e12-8dbf-549a7cca017c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:41.151138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:41.230731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-4930556e-9f01-4e12-8dbf-549a7cca017c None None] MFA Rules not processed for user `de05c08e30dd4fff93f1c97aac9553d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:41.232055 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-4930556e-9f01-4e12-8dbf-549a7cca017c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:41.326843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 662/1321] 10.222.0.61 () {58 vars in 1062 bytes} [Wed Sep 7 13:06:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:06:41.336978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890b138-b06b-4c5f-913e-d0df17e10c37 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:41.338613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5890b138-b06b-4c5f-913e-d0df17e10c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:41.466482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890b138-b06b-4c5f-913e-d0df17e10c37 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:41.555159 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb1287-4033-4f29-a666-ab32acc6b225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:41.556276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb1287-4033-4f29-a666-ab32acc6b225 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:41.556497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb1287-4033-4f29-a666-ab32acc6b225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:41.556911 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb1287-4033-4f29-a666-ab32acc6b225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:41.558699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-78bb1287-4033-4f29-a666-ab32acc6b225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:41.567925 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:41.758111 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb1287-4033-4f29-a666-ab32acc6b225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:41.761075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb1287-4033-4f29-a666-ab32acc6b225 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:41.800242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 660/1322] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:45.411472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d1e0d-e64c-4120-a21c-b340d4882608 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:45.412327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d1e0d-e64c-4120-a21c-b340d4882608 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:45.412327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d1e0d-e64c-4120-a21c-b340d4882608 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:45.414286 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 663/1323] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:45.423201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b1257-5c9f-475f-91de-f870c4b44402 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:45.423976 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b1257-5c9f-475f-91de-f870c4b44402 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:45.423976 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b1257-5c9f-475f-91de-f870c4b44402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:45.512747 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-cd8b1257-5c9f-475f-91de-f870c4b44402 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:45.514808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-cd8b1257-5c9f-475f-91de-f870c4b44402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:45.629730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:45.709938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 661/1324] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:46.022365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d1e0d-e64c-4120-a21c-b340d4882608 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:46.025453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a40d1e0d-e64c-4120-a21c-b340d4882608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:46.177335 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d1e0d-e64c-4120-a21c-b340d4882608 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:46.194183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:46.281965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518f1a7-7db0-44c3-868a-a6d5bd4ce945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:46.282536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518f1a7-7db0-44c3-868a-a6d5bd4ce945 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:46.282883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518f1a7-7db0-44c3-868a-a6d5bd4ce945 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:46.283264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518f1a7-7db0-44c3-868a-a6d5bd4ce945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:46.285278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3518f1a7-7db0-44c3-868a-a6d5bd4ce945 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:46.436547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518f1a7-7db0-44c3-868a-a6d5bd4ce945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:46.438045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518f1a7-7db0-44c3-868a-a6d5bd4ce945 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:46.461183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 664/1325] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:48.895815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eabdf1-e52a-4d55-8261-8cbeb2bcbe4f None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:48.896368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eabdf1-e52a-4d55-8261-8cbeb2bcbe4f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:48.896654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eabdf1-e52a-4d55-8261-8cbeb2bcbe4f None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:48.899412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 662/1326] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:06:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:48.908251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd842d8-a4a9-44ff-b6c0-460f5f8c8c09 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:48.908935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd842d8-a4a9-44ff-b6c0-460f5f8c8c09 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:48.908935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd842d8-a4a9-44ff-b6c0-460f5f8c8c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:48.969706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:49.003208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-5bd842d8-a4a9-44ff-b6c0-460f5f8c8c09 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:49.005452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5bd842d8-a4a9-44ff-b6c0-460f5f8c8c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:49.165264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 665/1327] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:49.484501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eabdf1-e52a-4d55-8261-8cbeb2bcbe4f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:49.486379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d9eabdf1-e52a-4d55-8261-8cbeb2bcbe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:49.642893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eabdf1-e52a-4d55-8261-8cbeb2bcbe4f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:49.753032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f88d25-1b4a-4fb4-a688-f20025da72c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:49.753641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f88d25-1b4a-4fb4-a688-f20025da72c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:49.754036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f88d25-1b4a-4fb4-a688-f20025da72c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:49.754384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f88d25-1b4a-4fb4-a688-f20025da72c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:49.756423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-78f88d25-1b4a-4fb4-a688-f20025da72c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:49.778568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:49.962835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f88d25-1b4a-4fb4-a688-f20025da72c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:49.965641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f88d25-1b4a-4fb4-a688-f20025da72c4 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:50.003388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 663/1328] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:06:52.974165 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b1bf8-ddd8-439c-bda0-c082f1f27210 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:52.974808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b1bf8-ddd8-439c-bda0-c082f1f27210 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:52.974808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b1bf8-ddd8-439c-bda0-c082f1f27210 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:52.976895 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 666/1329] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:06:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:06:52.984549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d8d58-f526-47d4-a695-ef05b77af853 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:52.985367 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d8d58-f526-47d4-a695-ef05b77af853 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:52.985553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d8d58-f526-47d4-a695-ef05b77af853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:53.075397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-1f6d8d58-f526-47d4-a695-ef05b77af853 None None] MFA Rules not processed for user `d7b269b260eb4a96aa4bc25ce7962e27`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:53.076675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1f6d8d58-f526-47d4-a695-ef05b77af853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:53.160078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:53.218073 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 664/1330] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:06:52 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:06:53.757848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b1bf8-ddd8-439c-bda0-c082f1f27210 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:53.759473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f00b1bf8-ddd8-439c-bda0-c082f1f27210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:53.893234 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b1bf8-ddd8-439c-bda0-c082f1f27210 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:53.919532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:54.015723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023ba8b-2bfb-49d6-aa3c-1698d1df7e1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:54.016710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023ba8b-2bfb-49d6-aa3c-1698d1df7e1e service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:54.017146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023ba8b-2bfb-49d6-aa3c-1698d1df7e1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:54.017426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023ba8b-2bfb-49d6-aa3c-1698d1df7e1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:54.020397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0023ba8b-2bfb-49d6-aa3c-1698d1df7e1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:54.212738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023ba8b-2bfb-49d6-aa3c-1698d1df7e1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:54.215138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023ba8b-2bfb-49d6-aa3c-1698d1df7e1e service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:54.241578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 667/1331] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:53 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:06:57.008490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca35d3-62ab-4508-ba59-68ceb4d56271 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:57.009018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca35d3-62ab-4508-ba59-68ceb4d56271 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:57.009018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca35d3-62ab-4508-ba59-68ceb4d56271 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:57.010566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 665/1332] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:06:57.016953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e818e3d-483d-46fc-aad2-e7611a99bdc4 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:57.017531 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e818e3d-483d-46fc-aad2-e7611a99bdc4 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:57.017531 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e818e3d-483d-46fc-aad2-e7611a99bdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:57.098902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:57.102896 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-1e818e3d-483d-46fc-aad2-e7611a99bdc4 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:06:57.104686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1e818e3d-483d-46fc-aad2-e7611a99bdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:57.238856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 668/1333] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:06:57.541930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca35d3-62ab-4508-ba59-68ceb4d56271 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:06:57.544996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-70ca35d3-62ab-4508-ba59-68ceb4d56271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:57.721851 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca35d3-62ab-4508-ba59-68ceb4d56271 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:06:57.810161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bdf3b-ea1a-45ed-9e24-3d5d766b8744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:06:57.810928 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bdf3b-ea1a-45ed-9e24-3d5d766b8744 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:06:57.811248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bdf3b-ea1a-45ed-9e24-3d5d766b8744 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:06:57.811528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bdf3b-ea1a-45ed-9e24-3d5d766b8744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:06:57.813255 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4d1bdf3b-ea1a-45ed-9e24-3d5d766b8744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:06:57.846385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:06:57.945522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bdf3b-ea1a-45ed-9e24-3d5d766b8744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:06:57.947007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bdf3b-ea1a-45ed-9e24-3d5d766b8744 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:06:57.969542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 666/1334] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:00.805549 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb169e-201b-473f-a535-f4e0848ecfcf None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:00.806214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb169e-201b-473f-a535-f4e0848ecfcf None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:00.806214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb169e-201b-473f-a535-f4e0848ecfcf None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:00.807486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 669/1335] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:00.813264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29bbf6-bd01-4cfc-81aa-5309be7625ec None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:00.813803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29bbf6-bd01-4cfc-81aa-5309be7625ec None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:00.813803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29bbf6-bd01-4cfc-81aa-5309be7625ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:00.891522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-9f29bbf6-bd01-4cfc-81aa-5309be7625ec None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:00.892776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9f29bbf6-bd01-4cfc-81aa-5309be7625ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:00.974067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:00.998170 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 667/1336] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:01.282236 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb169e-201b-473f-a535-f4e0848ecfcf None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:01.283840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8bfb169e-201b-473f-a535-f4e0848ecfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:01.409746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb169e-201b-473f-a535-f4e0848ecfcf None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:01.441599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:01.493686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9e5d9-8e1d-4e47-8ebf-12081066d597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:01.494293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9e5d9-8e1d-4e47-8ebf-12081066d597 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.494565 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9e5d9-8e1d-4e47-8ebf-12081066d597 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.495092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9e5d9-8e1d-4e47-8ebf-12081066d597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.496753 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-eca9e5d9-8e1d-4e47-8ebf-12081066d597 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:01.621301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9e5d9-8e1d-4e47-8ebf-12081066d597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:01.623050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9e5d9-8e1d-4e47-8ebf-12081066d597 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:01.642988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 670/1337] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:05.915566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00736ba5-63cf-48d5-94ef-9692f4bd116b None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:05.916542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00736ba5-63cf-48d5-94ef-9692f4bd116b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:05.916542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00736ba5-63cf-48d5-94ef-9692f4bd116b None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:05.918161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 668/1338] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:05.924610 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38355f15-8b16-4e62-bae3-9b3949ebffeb None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:05.925264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38355f15-8b16-4e62-bae3-9b3949ebffeb None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:05.925264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38355f15-8b16-4e62-bae3-9b3949ebffeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:06.013670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-38355f15-8b16-4e62-bae3-9b3949ebffeb None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:06.015265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-38355f15-8b16-4e62-bae3-9b3949ebffeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:06.027354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:06.161028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 671/1339] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:06.691019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00736ba5-63cf-48d5-94ef-9692f4bd116b None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:06.692689 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-00736ba5-63cf-48d5-94ef-9692f4bd116b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:06.899924 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00736ba5-63cf-48d5-94ef-9692f4bd116b None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:07.052278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76a001-e9e7-4f78-b420-fe688e829f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:07.055094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a001-e9e7-4f78-b420-fe688e829f93 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:07.055651 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a001-e9e7-4f78-b420-fe688e829f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:07.056175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a001-e9e7-4f78-b420-fe688e829f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:07.058991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ad76a001-e9e7-4f78-b420-fe688e829f93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:07.133161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:07.240780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76a001-e9e7-4f78-b420-fe688e829f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:07.242600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76a001-e9e7-4f78-b420-fe688e829f93 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:07.265512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 669/1340] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:12.340537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9d678-6c9d-4357-9237-5b79c3817689 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.341340 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9d678-6c9d-4357-9237-5b79c3817689 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.341340 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9d678-6c9d-4357-9237-5b79c3817689 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.343225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 672/1341] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:07:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:12.352195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a3293-086b-42aa-bf71-775dd9d4ee77 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:12.353071 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a3293-086b-42aa-bf71-775dd9d4ee77 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:12.353227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a3293-086b-42aa-bf71-775dd9d4ee77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:12.455454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-3c8a3293-086b-42aa-bf71-775dd9d4ee77 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:12.456858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3c8a3293-086b-42aa-bf71-775dd9d4ee77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:12.591904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:12.611690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 670/1342] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:12.966634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9d678-6c9d-4357-9237-5b79c3817689 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:12.969603 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-61b9d678-6c9d-4357-9237-5b79c3817689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:13.173684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9d678-6c9d-4357-9237-5b79c3817689 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:13.242838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:13.298729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff80c8-ad13-438d-9de9-a5cc32c31fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:13.299592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff80c8-ad13-438d-9de9-a5cc32c31fea service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:13.300018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff80c8-ad13-438d-9de9-a5cc32c31fea service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:13.300384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff80c8-ad13-438d-9de9-a5cc32c31fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:13.302248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e8ff80c8-ad13-438d-9de9-a5cc32c31fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:13.503245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff80c8-ad13-438d-9de9-a5cc32c31fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:13.504867 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff80c8-ad13-438d-9de9-a5cc32c31fea service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:13.536602 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 673/1343] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:17.243548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b8a37-1eab-4895-9c77-05f71aadda8f None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.244652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b8a37-1eab-4895-9c77-05f71aadda8f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.245623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b8a37-1eab-4895-9c77-05f71aadda8f None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.248473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 671/1344] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:07:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:17.258084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470a6e9-a51e-48d8-8ff8-2b9b27828749 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.258893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470a6e9-a51e-48d8-8ff8-2b9b27828749 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.259045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470a6e9-a51e-48d8-8ff8-2b9b27828749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.357953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8470a6e9-a51e-48d8-8ff8-2b9b27828749 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:17.359556 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8470a6e9-a51e-48d8-8ff8-2b9b27828749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:17.392155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:17.556197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 674/1345] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:18.067314 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b8a37-1eab-4895-9c77-05f71aadda8f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:18.070369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-360b8a37-1eab-4895-9c77-05f71aadda8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:18.242326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b8a37-1eab-4895-9c77-05f71aadda8f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:18.385588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c58d08-b0c1-4a91-b147-63896e7b4ab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:18.385588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c58d08-b0c1-4a91-b147-63896e7b4ab3 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:18.385588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c58d08-b0c1-4a91-b147-63896e7b4ab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:18.385588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c58d08-b0c1-4a91-b147-63896e7b4ab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:18.395047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f1c58d08-b0c1-4a91-b147-63896e7b4ab3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:18.496082 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:18.611532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c58d08-b0c1-4a91-b147-63896e7b4ab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:18.614320 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c58d08-b0c1-4a91-b147-63896e7b4ab3 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:18.647169 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 672/1346] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:24.929946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d36fc-5006-4b0f-8dd4-ce5a22924209 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:24.930517 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d36fc-5006-4b0f-8dd4-ce5a22924209 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:24.930517 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d36fc-5006-4b0f-8dd4-ce5a22924209 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:24.932403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 675/1347] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:07:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:24.938836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e8343-fec1-431a-9eec-6b06b0366098 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:24.939705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e8343-fec1-431a-9eec-6b06b0366098 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:24.939705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e8343-fec1-431a-9eec-6b06b0366098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:25.026300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-483e8343-fec1-431a-9eec-6b06b0366098 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:25.027903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-483e8343-fec1-431a-9eec-6b06b0366098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:25.165072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:25.169185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 673/1348] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:07:25.492863 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d36fc-5006-4b0f-8dd4-ce5a22924209 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:25.494469 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-4c5d36fc-5006-4b0f-8dd4-ce5a22924209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:25.634295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d36fc-5006-4b0f-8dd4-ce5a22924209 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:25.704471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:25.741037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05f43f-a1a7-4ffc-85f7-ccdb079f121d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:25.741561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05f43f-a1a7-4ffc-85f7-ccdb079f121d service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:25.741834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05f43f-a1a7-4ffc-85f7-ccdb079f121d service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:25.742105 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05f43f-a1a7-4ffc-85f7-ccdb079f121d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:25.743885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6d05f43f-a1a7-4ffc-85f7-ccdb079f121d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:25.910910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05f43f-a1a7-4ffc-85f7-ccdb079f121d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:25.912796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05f43f-a1a7-4ffc-85f7-ccdb079f121d service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:25.935967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 676/1349] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:07:32.561672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52823f-5a17-4ccb-8680-8c00a8e0ca84 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:32.562693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52823f-5a17-4ccb-8680-8c00a8e0ca84 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:32.563132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52823f-5a17-4ccb-8680-8c00a8e0ca84 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:32.570398 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 674/1350] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:07:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:32.576330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc07656-b68c-4b0c-87bc-3b6cedc825b7 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:32.576872 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc07656-b68c-4b0c-87bc-3b6cedc825b7 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:32.578872 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc07656-b68c-4b0c-87bc-3b6cedc825b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:32.795619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-0dc07656-b68c-4b0c-87bc-3b6cedc825b7 None None] MFA Rules not processed for user `9e283e62741f40439d02e545a212807a`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:32.795619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0dc07656-b68c-4b0c-87bc-3b6cedc825b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:32.850517 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:33.005156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 677/1351] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:07:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:38.762763 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7fa32-f312-44d4-87ca-6467c29a34ed None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:38.763500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7fa32-f312-44d4-87ca-6467c29a34ed None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:38.763500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7fa32-f312-44d4-87ca-6467c29a34ed None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:38.764942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 675/1352] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:38.773591 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830e8e2-82b9-43f9-a582-3e099f864a2e None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:38.774367 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830e8e2-82b9-43f9-a582-3e099f864a2e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:38.774367 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830e8e2-82b9-43f9-a582-3e099f864a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:38.884415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-9830e8e2-82b9-43f9-a582-3e099f864a2e None None] MFA Rules not processed for user `9e283e62741f40439d02e545a212807a`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:38.885973 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-9830e8e2-82b9-43f9-a582-3e099f864a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:39.016141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:39.086089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 678/1353] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:07:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:07:40.989476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f46297-eb36-40d4-aca6-777edf60b23f None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:40.990233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f46297-eb36-40d4-aca6-777edf60b23f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:40.990600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f46297-eb36-40d4-aca6-777edf60b23f None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:40.993393 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 676/1354] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:07:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:41.001658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574a4dd-a33a-4f8b-a2bd-622db87d3480 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:41.002472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574a4dd-a33a-4f8b-a2bd-622db87d3480 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:41.002472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574a4dd-a33a-4f8b-a2bd-622db87d3480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:41.115609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-2574a4dd-a33a-4f8b-a2bd-622db87d3480 None None] MFA Rules not processed for user `9e283e62741f40439d02e545a212807a`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:41.117749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2574a4dd-a33a-4f8b-a2bd-622db87d3480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:41.349873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 679/1355] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:07:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:45.656121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36d8e7-0ed0-44bb-9661-2c5a952577e8 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:45.656675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36d8e7-0ed0-44bb-9661-2c5a952577e8 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:45.657025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36d8e7-0ed0-44bb-9661-2c5a952577e8 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:45.659324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 677/1356] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:07:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:45.665743 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403c521-08e4-42b4-bf01-f580c4ab844b None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:45.666361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403c521-08e4-42b4-bf01-f580c4ab844b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:45.666361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403c521-08e4-42b4-bf01-f580c4ab844b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:45.710288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:45.751116 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-0403c521-08e4-42b4-bf01-f580c4ab844b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:45.752440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0403c521-08e4-42b4-bf01-f580c4ab844b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:45.874886 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 680/1357] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:46.183262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da095-affd-4f9e-a7fa-03bafabfeae0 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:46.184571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da095-affd-4f9e-a7fa-03bafabfeae0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:46.185647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da095-affd-4f9e-a7fa-03bafabfeae0 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:46.191305 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 678/1358] 10.222.0.61 () {54 vars in 962 bytes} [Wed Sep 7 13:07:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:46.200573 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e972e35-c46c-40ce-a481-670e6b3b19e4 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:46.202568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e972e35-c46c-40ce-a481-670e6b3b19e4 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:46.203428 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e972e35-c46c-40ce-a481-670e6b3b19e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:46.319125 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-5e972e35-c46c-40ce-a481-670e6b3b19e4 None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:46.320729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5e972e35-c46c-40ce-a481-670e6b3b19e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:46.388206 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:46.479081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 681/1359] 10.222.0.61 () {58 vars in 1059 bytes} [Wed Sep 7 13:07:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:46.493525 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da095-affd-4f9e-a7fa-03bafabfeae0 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:46.497050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-929da095-affd-4f9e-a7fa-03bafabfeae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:46.819299 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da095-affd-4f9e-a7fa-03bafabfeae0 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:47.000675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:47.098258 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba5477-51b2-42d7-b0fb-fb06d23908d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:47.099904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba5477-51b2-42d7-b0fb-fb06d23908d9 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:47.100518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba5477-51b2-42d7-b0fb-fb06d23908d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:47.101334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba5477-51b2-42d7-b0fb-fb06d23908d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:47.104726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5cba5477-51b2-42d7-b0fb-fb06d23908d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:47.278188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba5477-51b2-42d7-b0fb-fb06d23908d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:47.281687 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba5477-51b2-42d7-b0fb-fb06d23908d9 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:47.318433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 679/1360] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 825 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:51.040554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccab5e3-95ea-44d0-bd41-577f079471c5 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.041468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccab5e3-95ea-44d0-bd41-577f079471c5 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.041618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccab5e3-95ea-44d0-bd41-577f079471c5 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.043698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 682/1361] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:51.050567 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6ee7c-0a34-4272-8047-f91afebfc4cf None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.051262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6ee7c-0a34-4272-8047-f91afebfc4cf None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.051262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6ee7c-0a34-4272-8047-f91afebfc4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.157018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-05e6ee7c-0a34-4272-8047-f91afebfc4cf None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:51.158281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-05e6ee7c-0a34-4272-8047-f91afebfc4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:51.218115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:51.314582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 680/1362] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:07:51.636685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccab5e3-95ea-44d0-bd41-577f079471c5 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:51.638582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-cccab5e3-95ea-44d0-bd41-577f079471c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:51.813807 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccab5e3-95ea-44d0-bd41-577f079471c5 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:51.825417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:51.964736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae9011-2062-4086-8cfd-0d943e6af067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:51.966227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae9011-2062-4086-8cfd-0d943e6af067 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.966873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae9011-2062-4086-8cfd-0d943e6af067 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.967633 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae9011-2062-4086-8cfd-0d943e6af067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.970752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f9ae9011-2062-4086-8cfd-0d943e6af067 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:52.088530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae9011-2062-4086-8cfd-0d943e6af067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:52.090302 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae9011-2062-4086-8cfd-0d943e6af067 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:52.112264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 683/1363] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:54.805985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793e600-16d9-44cd-bdb8-84e0c02fc99b None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:54.806540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793e600-16d9-44cd-bdb8-84e0c02fc99b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:54.806980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793e600-16d9-44cd-bdb8-84e0c02fc99b None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:54.809941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 681/1364] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:07:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:54.817212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14f346-9f9a-471d-a774-529c3defc2ff None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:54.817917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14f346-9f9a-471d-a774-529c3defc2ff None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:54.818471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14f346-9f9a-471d-a774-529c3defc2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:54.849371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:54.948642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ea14f346-9f9a-471d-a774-529c3defc2ff None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:54.950028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ea14f346-9f9a-471d-a774-529c3defc2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:55.091640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 684/1365] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:55.388586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f314dd-042c-4c47-8c29-6823f82fcb32 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:55.389222 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f314dd-042c-4c47-8c29-6823f82fcb32 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:55.389810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f314dd-042c-4c47-8c29-6823f82fcb32 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:55.393885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 682/1366] 10.222.0.61 () {54 vars in 962 bytes} [Wed Sep 7 13:07:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:55.403525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641d6db-330d-4b5b-9f85-7ada128ea763 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:55.403858 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641d6db-330d-4b5b-9f85-7ada128ea763 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:55.404119 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641d6db-330d-4b5b-9f85-7ada128ea763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:55.513259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-1641d6db-330d-4b5b-9f85-7ada128ea763 None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:55.514849 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1641d6db-330d-4b5b-9f85-7ada128ea763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:55.563348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:55.688099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 685/1367] 10.222.0.61 () {58 vars in 1059 bytes} [Wed Sep 7 13:07:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:55.703030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f314dd-042c-4c47-8c29-6823f82fcb32 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:55.705970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-13f314dd-042c-4c47-8c29-6823f82fcb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:55.991909 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f314dd-042c-4c47-8c29-6823f82fcb32 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:56.033949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:56.247869 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf691861-ef95-475f-bb98-e03fce53368e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:56.248856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf691861-ef95-475f-bb98-e03fce53368e service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:56.249356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf691861-ef95-475f-bb98-e03fce53368e service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:56.249845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf691861-ef95-475f-bb98-e03fce53368e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:56.253477 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bf691861-ef95-475f-bb98-e03fce53368e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:56.415058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df43a0-2959-463c-98b2-7a2090ca95b0 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:56.415957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df43a0-2959-463c-98b2-7a2090ca95b0 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:56.415957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df43a0-2959-463c-98b2-7a2090ca95b0 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:56.417139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 686/1368] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:07:56.422903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b29556-9112-42a5-9b7f-55516c996597 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:56.423252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b29556-9112-42a5-9b7f-55516c996597 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:56.423606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b29556-9112-42a5-9b7f-55516c996597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:56.463187 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf691861-ef95-475f-bb98-e03fce53368e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:56.466536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf691861-ef95-475f-bb98-e03fce53368e service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:56.504524 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 683/1369] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 803 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:07:56.520133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-58b29556-9112-42a5-9b7f-55516c996597 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:56.522535 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-58b29556-9112-42a5-9b7f-55516c996597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:56.677222 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:56.687192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 687/1370] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:07:57.104762 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f88c3-320a-43da-ac4d-171f807eb23a None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:57.105797 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f88c3-320a-43da-ac4d-171f807eb23a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:57.105797 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f88c3-320a-43da-ac4d-171f807eb23a None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:57.109164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 684/1371] 10.222.0.61 () {54 vars in 964 bytes} [Wed Sep 7 13:07:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:57.118461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c48cd-58b5-46df-8753-894a314d14fa None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:57.118905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c48cd-58b5-46df-8753-894a314d14fa None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:57.120067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c48cd-58b5-46df-8753-894a314d14fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:57.297713 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-5b3c48cd-58b5-46df-8753-894a314d14fa None None] MFA Rules not processed for user `8356f60a0f684484abc8f8d6fb6a7be6`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:57.300663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5b3c48cd-58b5-46df-8753-894a314d14fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:57.437082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 688/1372] 10.222.0.61 () {58 vars in 1061 bytes} [Wed Sep 7 13:07:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:57.447030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f88c3-320a-43da-ac4d-171f807eb23a None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:57.448977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-830f88c3-320a-43da-ac4d-171f807eb23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:57.496366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:57.637459 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f88c3-320a-43da-ac4d-171f807eb23a None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:57.764644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce46aaa-ff10-4ec9-992c-b7ae92d957ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:57.765265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce46aaa-ff10-4ec9-992c-b7ae92d957ce service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:57.765549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce46aaa-ff10-4ec9-992c-b7ae92d957ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:57.765819 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce46aaa-ff10-4ec9-992c-b7ae92d957ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:57.767557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-dce46aaa-ff10-4ec9-992c-b7ae92d957ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:57.882140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:57.951582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce46aaa-ff10-4ec9-992c-b7ae92d957ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:57.954339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce46aaa-ff10-4ec9-992c-b7ae92d957ce service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:57.992991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 685/1373] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:07:59.416451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a2c93-b20a-490f-bc8c-3077e39d9177 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:59.417011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a2c93-b20a-490f-bc8c-3077e39d9177 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:59.417292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a2c93-b20a-490f-bc8c-3077e39d9177 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:59.418958 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 689/1374] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:07:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:07:59.425375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae14bc9-5c76-4fe6-b3b3-61dd426415be None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:59.426031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae14bc9-5c76-4fe6-b3b3-61dd426415be None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:59.426031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae14bc9-5c76-4fe6-b3b3-61dd426415be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:59.503796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-1ae14bc9-5c76-4fe6-b3b3-61dd426415be None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:59.505092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1ae14bc9-5c76-4fe6-b3b3-61dd426415be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:59.653164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 686/1375] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:07:59.961639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a2c93-b20a-490f-bc8c-3077e39d9177 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:59.963685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-be2a2c93-b20a-490f-bc8c-3077e39d9177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:00.089119 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:00.169799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a2c93-b20a-490f-bc8c-3077e39d9177 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:00.332003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c922e87-faf9-44d9-9971-639fde04a4c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:00.332638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c922e87-faf9-44d9-9971-639fde04a4c2 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:00.332910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c922e87-faf9-44d9-9971-639fde04a4c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:00.333241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c922e87-faf9-44d9-9971-639fde04a4c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:00.334959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5c922e87-faf9-44d9-9971-639fde04a4c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:00.403157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:00.452377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c922e87-faf9-44d9-9971-639fde04a4c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:00.454042 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c922e87-faf9-44d9-9971-639fde04a4c2 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:00.475919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 690/1376] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:07:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:02.998667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55996f-4eea-4c38-bb4b-703b7ea13400 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:02.999254 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55996f-4eea-4c38-bb4b-703b7ea13400 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:02.999254 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55996f-4eea-4c38-bb4b-703b7ea13400 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:03.001040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 687/1377] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:03.007002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753caca-a246-4dc1-a450-e015d1c5bafc None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:03.007557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753caca-a246-4dc1-a450-e015d1c5bafc None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:03.007557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753caca-a246-4dc1-a450-e015d1c5bafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:03.088994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-7753caca-a246-4dc1-a450-e015d1c5bafc None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:03.090229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7753caca-a246-4dc1-a450-e015d1c5bafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:03.210576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 691/1378] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:03.536534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55996f-4eea-4c38-bb4b-703b7ea13400 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:03.540013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ff55996f-4eea-4c38-bb4b-703b7ea13400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:03.595530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:03.782353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55996f-4eea-4c38-bb4b-703b7ea13400 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:04.001866 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:04.012865 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6def3d-f708-4a06-8612-277aeb1b10d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:04.015014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6def3d-f708-4a06-8612-277aeb1b10d8 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:04.015678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6def3d-f708-4a06-8612-277aeb1b10d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:04.016273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6def3d-f708-4a06-8612-277aeb1b10d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:04.019417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-8d6def3d-f708-4a06-8612-277aeb1b10d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:04.215210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6def3d-f708-4a06-8612-277aeb1b10d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:04.218577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6def3d-f708-4a06-8612-277aeb1b10d8 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:04.261234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 688/1379] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:07.107454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a024a9-4128-4650-bc4d-e283d3f56beb None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:07.107960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a024a9-4128-4650-bc4d-e283d3f56beb None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:07.108447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a024a9-4128-4650-bc4d-e283d3f56beb None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:07.110833 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 692/1380] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:08:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:07.119706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f2fa3-9382-4559-8434-5afccd03aca4 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:07.120396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f2fa3-9382-4559-8434-5afccd03aca4 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:07.120396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f2fa3-9382-4559-8434-5afccd03aca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:07.208387 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f93f2fa3-9382-4559-8434-5afccd03aca4 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:07.209805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f93f2fa3-9382-4559-8434-5afccd03aca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:07.314305 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:07.379777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 689/1381] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:07.928760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a024a9-4128-4650-bc4d-e283d3f56beb None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:07.931742 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-31a024a9-4128-4650-bc4d-e283d3f56beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:07.992393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:08.226593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a024a9-4128-4650-bc4d-e283d3f56beb None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:08.419592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:08.457790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233a564-5ddc-4bee-a359-571a52c90d2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:08.459526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233a564-5ddc-4bee-a359-571a52c90d2c service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:08.460049 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233a564-5ddc-4bee-a359-571a52c90d2c service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:08.460605 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233a564-5ddc-4bee-a359-571a52c90d2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:08.464049 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2233a564-5ddc-4bee-a359-571a52c90d2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:08.631404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233a564-5ddc-4bee-a359-571a52c90d2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:08.633636 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233a564-5ddc-4bee-a359-571a52c90d2c service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:08.663298 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 693/1382] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 736 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:11.519977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a727738-5df1-45df-996d-74ebf4942021 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:11.521604 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a727738-5df1-45df-996d-74ebf4942021 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:11.521604 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a727738-5df1-45df-996d-74ebf4942021 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:11.522417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 690/1383] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:08:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:11.529674 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301e746-273f-4be1-aa23-ff9dffcb2025 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:11.530568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301e746-273f-4be1-aa23-ff9dffcb2025 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:11.530715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301e746-273f-4be1-aa23-ff9dffcb2025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:11.653406 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-e301e746-273f-4be1-aa23-ff9dffcb2025 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:11.655859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e301e746-273f-4be1-aa23-ff9dffcb2025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:11.761431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:11.816147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 694/1384] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:08:12.265989 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a727738-5df1-45df-996d-74ebf4942021 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:12.267621 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7a727738-5df1-45df-996d-74ebf4942021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:12.480718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a727738-5df1-45df-996d-74ebf4942021 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:12.573599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:12.706532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744165a2-fc50-46fb-ac02-47b4cd0c4e5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:12.708011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744165a2-fc50-46fb-ac02-47b4cd0c4e5b service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:12.709201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744165a2-fc50-46fb-ac02-47b4cd0c4e5b service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:12.709821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744165a2-fc50-46fb-ac02-47b4cd0c4e5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:12.714319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-744165a2-fc50-46fb-ac02-47b4cd0c4e5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:12.887417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744165a2-fc50-46fb-ac02-47b4cd0c4e5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:12.890447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744165a2-fc50-46fb-ac02-47b4cd0c4e5b service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:12.919808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 691/1385] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:15.199940 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968c4d6-1d34-4162-aa52-bc404937efa2 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:15.200706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968c4d6-1d34-4162-aa52-bc404937efa2 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:15.200706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968c4d6-1d34-4162-aa52-bc404937efa2 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:15.203323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 695/1386] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:08:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:15.210592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0644f53-3eda-4f21-8245-7c1bfd14b59e None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:15.211412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0644f53-3eda-4f21-8245-7c1bfd14b59e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:15.211412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0644f53-3eda-4f21-8245-7c1bfd14b59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:15.309237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f0644f53-3eda-4f21-8245-7c1bfd14b59e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:15.311453 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f0644f53-3eda-4f21-8245-7c1bfd14b59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:15.329582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:15.468564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 692/1387] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:08:15.787602 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d6b50-3538-43d4-900b-1a348b5fc51e None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:15.788089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d6b50-3538-43d4-900b-1a348b5fc51e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:15.788472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d6b50-3538-43d4-900b-1a348b5fc51e None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:15.790249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 696/1388] 10.222.0.61 () {54 vars in 961 bytes} [Wed Sep 7 13:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:15.797711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f1c1f-58a1-43ef-b5bd-095a8e804ea7 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:15.798215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f1c1f-58a1-43ef-b5bd-095a8e804ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:15.798732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f1c1f-58a1-43ef-b5bd-095a8e804ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:15.836722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c4be4-1678-4298-86d6-d1882e0474e7 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:15.837096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c4be4-1678-4298-86d6-d1882e0474e7 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:15.837455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c4be4-1678-4298-86d6-d1882e0474e7 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:15.838935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 697/1389] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:08:15.845846 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dec3f-ceb0-4436-be99-26f4f0cc4d51 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:15.846125 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dec3f-ceb0-4436-be99-26f4f0cc4d51 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:15.846384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dec3f-ceb0-4436-be99-26f4f0cc4d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:15.931468 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-033f1c1f-58a1-43ef-b5bd-095a8e804ea7 None None] MFA Rules not processed for user `b4f82adce75b44c6baf85b6b96dd0728`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:15.932760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-033f1c1f-58a1-43ef-b5bd-095a8e804ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:15.953242 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ea0dec3f-ceb0-4436-be99-26f4f0cc4d51 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:15.955131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ea0dec3f-ceb0-4436-be99-26f4f0cc4d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:16.022143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 693/1390] 10.222.0.61 () {58 vars in 1058 bytes} [Wed Sep 7 13:08:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:16.031667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f1c1f-58a1-43ef-b5bd-095a8e804ea7 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:16.036682 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:16.043489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f1c1f-58a1-43ef-b5bd-095a8e804ea7 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:16.100952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 698/1391] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:08:16.169730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75800579-5c94-49d9-85f7-1bcfab828e88 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f82adce75b44c6baf85b6b96dd0728', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:16.170514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75800579-5c94-49d9-85f7-1bcfab828e88 service glance] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:16.171015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75800579-5c94-49d9-85f7-1bcfab828e88 service glance] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:16.171545 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75800579-5c94-49d9-85f7-1bcfab828e88 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:16.174840 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-75800579-5c94-49d9-85f7-1bcfab828e88 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:16.339377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75800579-5c94-49d9-85f7-1bcfab828e88 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:16.340841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75800579-5c94-49d9-85f7-1bcfab828e88 service glance] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:16.375359 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 694/1392] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:08:16.402006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dec3f-ceb0-4436-be99-26f4f0cc4d51 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:16.403412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ea0dec3f-ceb0-4436-be99-26f4f0cc4d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:16.461168 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:16.548827 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dec3f-ceb0-4436-be99-26f4f0cc4d51 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:16.720330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f209b2f-d7a9-46ae-a4e5-14de5a4bdef2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:16.720871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f209b2f-d7a9-46ae-a4e5-14de5a4bdef2 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:16.721147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f209b2f-d7a9-46ae-a4e5-14de5a4bdef2 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:16.721404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f209b2f-d7a9-46ae-a4e5-14de5a4bdef2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:16.723078 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5f209b2f-d7a9-46ae-a4e5-14de5a4bdef2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:16.770025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:16.872394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f209b2f-d7a9-46ae-a4e5-14de5a4bdef2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:16.875139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f209b2f-d7a9-46ae-a4e5-14de5a4bdef2 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:16.902819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 699/1393] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:17.274987 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15c6cc-4e8f-44cd-abc0-f8929fb21c5f None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:17.274987 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15c6cc-4e8f-44cd-abc0-f8929fb21c5f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:17.276847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15c6cc-4e8f-44cd-abc0-f8929fb21c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:17.327081 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:17.365933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-dc15c6cc-4e8f-44cd-abc0-f8929fb21c5f None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:17.367204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-dc15c6cc-4e8f-44cd-abc0-f8929fb21c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:17.486022 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 695/1394] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:08:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:17.551090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781eff9-472a-40b5-a27c-1d678696ab2f None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:17.552415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781eff9-472a-40b5-a27c-1d678696ab2f None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:17.552608 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781eff9-472a-40b5-a27c-1d678696ab2f None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:17.554214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 700/1395] 10.222.0.61 () {54 vars in 943 bytes} [Wed Sep 7 13:08:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:08:17.563269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83d346-e700-4685-9f1b-3561a13bbb02 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:17.563773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83d346-e700-4685-9f1b-3561a13bbb02 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:17.564273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83d346-e700-4685-9f1b-3561a13bbb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:17.679720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-5a83d346-e700-4685-9f1b-3561a13bbb02 None None] MFA Rules not processed for user `0f886e0999d34bf9870b448772cc38a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:17.681176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5a83d346-e700-4685-9f1b-3561a13bbb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:17.756557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:17.841956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 696/1396] 10.222.0.61 () {58 vars in 1040 bytes} [Wed Sep 7 13:08:17 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:17.852054 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781eff9-472a-40b5-a27c-1d678696ab2f None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:17.853727 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f781eff9-472a-40b5-a27c-1d678696ab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:18.115240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781eff9-472a-40b5-a27c-1d678696ab2f None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:18.170609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:18.292985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921245de-37e4-4ce6-b670-f51b805070ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:18.294014 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921245de-37e4-4ce6-b670-f51b805070ec service swift] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:18.294451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921245de-37e4-4ce6-b670-f51b805070ec service swift] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:18.294929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921245de-37e4-4ce6-b670-f51b805070ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:18.298255 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-921245de-37e4-4ce6-b670-f51b805070ec service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:18.443655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921245de-37e4-4ce6-b670-f51b805070ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:18.445307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921245de-37e4-4ce6-b670-f51b805070ec service swift] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:18.455418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 701/1397] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:08:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:08:19.340256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b998c5-eb2d-433f-877a-0ae9dc861f1a None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:19.343852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b998c5-eb2d-433f-877a-0ae9dc861f1a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:19.343852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b998c5-eb2d-433f-877a-0ae9dc861f1a None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:19.343852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 697/1398] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:19.349674 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435f2e3-c531-4577-9eaa-b7f888777dda None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:19.350470 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435f2e3-c531-4577-9eaa-b7f888777dda None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:19.350470 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435f2e3-c531-4577-9eaa-b7f888777dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:19.464224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-5435f2e3-c531-4577-9eaa-b7f888777dda None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:19.465487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5435f2e3-c531-4577-9eaa-b7f888777dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:19.494997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:19.675272 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 702/1399] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:19.990756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b998c5-eb2d-433f-877a-0ae9dc861f1a None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:19.992407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-00b998c5-eb2d-433f-877a-0ae9dc861f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:20.227796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b998c5-eb2d-433f-877a-0ae9dc861f1a None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:20.292948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:20.429690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ccc54-722e-4940-ab43-7a8446455ba3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:20.430823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ccc54-722e-4940-ab43-7a8446455ba3 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:20.431156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ccc54-722e-4940-ab43-7a8446455ba3 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:20.431417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ccc54-722e-4940-ab43-7a8446455ba3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:20.433189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-814ccc54-722e-4940-ab43-7a8446455ba3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:20.566354 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ccc54-722e-4940-ab43-7a8446455ba3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:20.569027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ccc54-722e-4940-ab43-7a8446455ba3 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:20.601898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 698/1400] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:23.012549 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22edae-1d40-40b5-b7db-ff3e426ba6de None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:23.013171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22edae-1d40-40b5-b7db-ff3e426ba6de None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:23.013171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22edae-1d40-40b5-b7db-ff3e426ba6de None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:23.014701 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 703/1401] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:23.022147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413e3db-128f-4fa5-b2b7-8791ea912b35 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:23.022689 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413e3db-128f-4fa5-b2b7-8791ea912b35 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:23.023089 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413e3db-128f-4fa5-b2b7-8791ea912b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:23.106340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:23.145107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-1413e3db-128f-4fa5-b2b7-8791ea912b35 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:23.147389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1413e3db-128f-4fa5-b2b7-8791ea912b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:23.295265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 699/1402] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:23.652160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22edae-1d40-40b5-b7db-ff3e426ba6de None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:23.654368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ea22edae-1d40-40b5-b7db-ff3e426ba6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:23.915819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:23.925156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22edae-1d40-40b5-b7db-ff3e426ba6de None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:24.072989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d2177-3f01-48a5-9726-c56f737ac2db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:24.074487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d2177-3f01-48a5-9726-c56f737ac2db service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:24.075764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d2177-3f01-48a5-9726-c56f737ac2db service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:24.075764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d2177-3f01-48a5-9726-c56f737ac2db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:24.080187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-814d2177-3f01-48a5-9726-c56f737ac2db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:24.194936 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d2177-3f01-48a5-9726-c56f737ac2db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:24.196588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d2177-3f01-48a5-9726-c56f737ac2db service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:24.223028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:24.230408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 704/1403] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:26.835710 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907f151-7d26-4a9a-b2d3-346c8071c8f0 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:26.836541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907f151-7d26-4a9a-b2d3-346c8071c8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:26.836541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907f151-7d26-4a9a-b2d3-346c8071c8f0 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:26.842338 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 700/1404] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:08:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:26.845440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f00e7-6b72-4374-b1af-f93cffa0ee55 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:26.846816 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f00e7-6b72-4374-b1af-f93cffa0ee55 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:26.847777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f00e7-6b72-4374-b1af-f93cffa0ee55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:27.014878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-c66f00e7-6b72-4374-b1af-f93cffa0ee55 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:27.017144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c66f00e7-6b72-4374-b1af-f93cffa0ee55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:27.262126 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 705/1405] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:27.602810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907f151-7d26-4a9a-b2d3-346c8071c8f0 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:27.604736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1907f151-7d26-4a9a-b2d3-346c8071c8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:27.790095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:27.853157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907f151-7d26-4a9a-b2d3-346c8071c8f0 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:28.055339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07027cb9-3bfa-4601-9aaa-d9d762feb9b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:28.056635 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07027cb9-3bfa-4601-9aaa-d9d762feb9b0 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:28.057233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07027cb9-3bfa-4601-9aaa-d9d762feb9b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:28.057699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07027cb9-3bfa-4601-9aaa-d9d762feb9b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:28.061040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-07027cb9-3bfa-4601-9aaa-d9d762feb9b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:28.233052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:28.257216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07027cb9-3bfa-4601-9aaa-d9d762feb9b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:28.260374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07027cb9-3bfa-4601-9aaa-d9d762feb9b0 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:28.294929 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 701/1406] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:30.926497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa473a0b-90c2-4028-b97c-35e503b65a7d None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:30.927081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa473a0b-90c2-4028-b97c-35e503b65a7d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:30.927740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa473a0b-90c2-4028-b97c-35e503b65a7d None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:30.931242 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 706/1407] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:08:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:30.936226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546fba2-93f4-4f87-b343-8b8a13a49f26 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:30.936991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546fba2-93f4-4f87-b343-8b8a13a49f26 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:30.936991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546fba2-93f4-4f87-b343-8b8a13a49f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:31.033020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a546fba2-93f4-4f87-b343-8b8a13a49f26 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:31.035218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a546fba2-93f4-4f87-b343-8b8a13a49f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:31.202921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 702/1408] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:08:31.495326 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa473a0b-90c2-4028-b97c-35e503b65a7d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:31.498503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-aa473a0b-90c2-4028-b97c-35e503b65a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:31.607668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:31.721282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa473a0b-90c2-4028-b97c-35e503b65a7d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:31.918831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bef29a-95a0-44e3-9703-ec2889155e69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:31.919695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bef29a-95a0-44e3-9703-ec2889155e69 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:31.920219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bef29a-95a0-44e3-9703-ec2889155e69 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:31.920565 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bef29a-95a0-44e3-9703-ec2889155e69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:31.922388 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-66bef29a-95a0-44e3-9703-ec2889155e69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:31.986649 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:32.067989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bef29a-95a0-44e3-9703-ec2889155e69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:32.069973 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bef29a-95a0-44e3-9703-ec2889155e69 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:32.102047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 707/1409] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:41.195540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=75 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:42.098732 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=27 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:44.371317 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2654c-8550-49b9-935a-016b191de455 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:44.372009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2654c-8550-49b9-935a-016b191de455 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:44.372009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2654c-8550-49b9-935a-016b191de455 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:44.373336 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 703/1410] 10.222.0.61 () {54 vars in 918 bytes} [Wed Sep 7 13:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:44.380009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4b73a-86e5-4a82-b6bb-409b0ef344e7 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:44.380697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4b73a-86e5-4a82-b6bb-409b0ef344e7 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:44.380697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4b73a-86e5-4a82-b6bb-409b0ef344e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:44.477239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-98f4b73a-86e5-4a82-b6bb-409b0ef344e7 None None] MFA Rules not processed for user `9e283e62741f40439d02e545a212807a`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:44.478519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-98f4b73a-86e5-4a82-b6bb-409b0ef344e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:44.604193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 708/1411] 10.222.0.61 () {58 vars in 1015 bytes} [Wed Sep 7 13:08:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:08:44.624531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1354068-3321-4701-8c7e-264e015a69fc None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:44.624875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1354068-3321-4701-8c7e-264e015a69fc None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:44.625250 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1354068-3321-4701-8c7e-264e015a69fc None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:44.626795 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 704/1412] 10.222.0.61 () {54 vars in 918 bytes} [Wed Sep 7 13:08:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:44.634003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11d2e1-1a29-4c1a-a8f8-6f154f05878b None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:44.634724 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11d2e1-1a29-4c1a-a8f8-6f154f05878b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:44.635088 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11d2e1-1a29-4c1a-a8f8-6f154f05878b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:44.730095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:44.735997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-2c11d2e1-1a29-4c1a-a8f8-6f154f05878b None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:44.737522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2c11d2e1-1a29-4c1a-a8f8-6f154f05878b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:44.884010 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 709/1413] 10.222.0.61 () {58 vars in 1015 bytes} [Wed Sep 7 13:08:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:47.849725 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93a613-6ab7-45a6-98a1-310c9b09c60e None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:47.850445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93a613-6ab7-45a6-98a1-310c9b09c60e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:47.850445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93a613-6ab7-45a6-98a1-310c9b09c60e None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:47.851692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:47.858937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacda5f7-e230-4c3e-88d4-06887770efda None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:47.859590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacda5f7-e230-4c3e-88d4-06887770efda None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:47.859939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacda5f7-e230-4c3e-88d4-06887770efda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:47.949000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-aacda5f7-e230-4c3e-88d4-06887770efda None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:47.950285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-aacda5f7-e230-4c3e-88d4-06887770efda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:48.015024 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:48.067010 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:48.411016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93a613-6ab7-45a6-98a1-310c9b09c60e None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:48.413554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-8a93a613-6ab7-45a6-98a1-310c9b09c60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:48.591891 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93a613-6ab7-45a6-98a1-310c9b09c60e None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:48.679331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:48.726257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa9708-5b8e-4a0b-bf75-acdf096ca63d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:48.726884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa9708-5b8e-4a0b-bf75-acdf096ca63d service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:48.727196 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa9708-5b8e-4a0b-bf75-acdf096ca63d service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:48.727471 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa9708-5b8e-4a0b-bf75-acdf096ca63d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:48.729189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-35aa9708-5b8e-4a0b-bf75-acdf096ca63d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:48.845368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa9708-5b8e-4a0b-bf75-acdf096ca63d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:48.847008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa9708-5b8e-4a0b-bf75-acdf096ca63d service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:48.870111 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 706/1416] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:08:50.099120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c91180-f520-417c-bfa8-899c8294fd29 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:50.099712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c91180-f520-417c-bfa8-899c8294fd29 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:50.099893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c91180-f520-417c-bfa8-899c8294fd29 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:50.102279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:08:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:50.109725 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8648423-3220-4f15-9ce2-f0c9c40018f3 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:50.110326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8648423-3220-4f15-9ce2-f0c9c40018f3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:50.110326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8648423-3220-4f15-9ce2-f0c9c40018f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:50.222281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-d8648423-3220-4f15-9ce2-f0c9c40018f3 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:50.224967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d8648423-3220-4f15-9ce2-f0c9c40018f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:50.280068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:50.380713 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:50.583664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c91180-f520-417c-bfa8-899c8294fd29 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:50.585327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a7c91180-f520-417c-bfa8-899c8294fd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:50.760410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c91180-f520-417c-bfa8-899c8294fd29 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:50.808522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:50.908571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75e926-9d6c-40b5-a61c-a85eb9dc9b7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:50.909271 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75e926-9d6c-40b5-a61c-a85eb9dc9b7d service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:50.909593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75e926-9d6c-40b5-a61c-a85eb9dc9b7d service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:50.910425 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75e926-9d6c-40b5-a61c-a85eb9dc9b7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:50.912394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7f75e926-9d6c-40b5-a61c-a85eb9dc9b7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:51.020230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75e926-9d6c-40b5-a61c-a85eb9dc9b7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:51.021777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75e926-9d6c-40b5-a61c-a85eb9dc9b7d service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:51.043905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 712/1419] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:08:52.298605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fc0df-f1f7-4415-a7b4-2ce788f8e4c0 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.299284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fc0df-f1f7-4415-a7b4-2ce788f8e4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.299284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fc0df-f1f7-4415-a7b4-2ce788f8e4c0 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.301171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:08:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:08:52.310066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b7280-2c57-4f9d-af4b-11b015dc282c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.310898 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b7280-2c57-4f9d-af4b-11b015dc282c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.310898 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b7280-2c57-4f9d-af4b-11b015dc282c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.377729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:52.404740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-414b7280-2c57-4f9d-af4b-11b015dc282c None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:52.405997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-414b7280-2c57-4f9d-af4b-11b015dc282c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:52.518828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:08:52.802749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0fc0df-f1f7-4415-a7b4-2ce788f8e4c0 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:52.804345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4d0fc0df-f1f7-4415-a7b4-2ce788f8e4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:52.984294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0fc0df-f1f7-4415-a7b4-2ce788f8e4c0 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:52.999085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:53.116245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b102b9-63e8-483b-962f-3babe68b06f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:53.116886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b102b9-63e8-483b-962f-3babe68b06f7 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:53.117166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b102b9-63e8-483b-962f-3babe68b06f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:53.117460 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b102b9-63e8-483b-962f-3babe68b06f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:53.119500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-01b102b9-63e8-483b-962f-3babe68b06f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:53.226782 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b102b9-63e8-483b-962f-3babe68b06f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:53.228424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b102b9-63e8-483b-962f-3babe68b06f7 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:53.249919 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 709/1422] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:54.489470 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abce6e-c6a6-445f-8b1c-920982da5b9a None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.490356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abce6e-c6a6-445f-8b1c-920982da5b9a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.490543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abce6e-c6a6-445f-8b1c-920982da5b9a None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.492112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:08:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:54.499045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b69525-94ca-4c64-8822-fb7df937a2d7 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.499627 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b69525-94ca-4c64-8822-fb7df937a2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.499740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b69525-94ca-4c64-8822-fb7df937a2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.520399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:54.581329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-14b69525-94ca-4c64-8822-fb7df937a2d7 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:54.583380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-14b69525-94ca-4c64-8822-fb7df937a2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:54.690969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:54.847016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abce6e-c6a6-445f-8b1c-920982da5b9a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.848678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-33abce6e-c6a6-445f-8b1c-920982da5b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:54.979012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:55.019009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abce6e-c6a6-445f-8b1c-920982da5b9a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:55.145006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093620fa-881d-4685-bd06-881cd75b26e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:55.145624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093620fa-881d-4685-bd06-881cd75b26e7 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:55.145901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093620fa-881d-4685-bd06-881cd75b26e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:55.146168 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093620fa-881d-4685-bd06-881cd75b26e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:55.147851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-093620fa-881d-4685-bd06-881cd75b26e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:55.234548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:55.249459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093620fa-881d-4685-bd06-881cd75b26e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:55.251118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093620fa-881d-4685-bd06-881cd75b26e7 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:55.271786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 715/1425] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:56.514830 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf20e-df26-4eaf-bbaf-58f1d6e45d0c None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:56.515501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf20e-df26-4eaf-bbaf-58f1d6e45d0c None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:56.515706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf20e-df26-4eaf-bbaf-58f1d6e45d0c None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:56.517204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:56.524650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a1dd5-448f-47b0-ba68-85e21c244079 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:56.525471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a1dd5-448f-47b0-ba68-85e21c244079 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:56.525663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a1dd5-448f-47b0-ba68-85e21c244079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:56.619557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-844a1dd5-448f-47b0-ba68-85e21c244079 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:56.621593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-844a1dd5-448f-47b0-ba68-85e21c244079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:56.766745 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:08:57.049833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccf20e-df26-4eaf-bbaf-58f1d6e45d0c None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:57.053496 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f1ccf20e-df26-4eaf-bbaf-58f1d6e45d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:57.161016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:57.237001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccf20e-df26-4eaf-bbaf-58f1d6e45d0c None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:57.399684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf04ab8-1dd7-4bd0-a0e5-a059fa76be9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:57.400214 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf04ab8-1dd7-4bd0-a0e5-a059fa76be9b service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:57.400492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf04ab8-1dd7-4bd0-a0e5-a059fa76be9b service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:57.400759 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf04ab8-1dd7-4bd0-a0e5-a059fa76be9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:57.402457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1cf04ab8-1dd7-4bd0-a0e5-a059fa76be9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:57.459497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:57.505241 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf04ab8-1dd7-4bd0-a0e5-a059fa76be9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:57.506915 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf04ab8-1dd7-4bd0-a0e5-a059fa76be9b service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:57.528431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 712/1428] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:08:58.675652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4ea11-d2ad-4ff2-ad34-0b24bbd92b54 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:58.676441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4ea11-d2ad-4ff2-ad34-0b24bbd92b54 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:58.676441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4ea11-d2ad-4ff2-ad34-0b24bbd92b54 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:58.677476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:08:58.684947 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fddd58-9533-4e88-9295-f3207948ba84 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:58.685603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fddd58-9533-4e88-9295-f3207948ba84 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:58.685886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fddd58-9533-4e88-9295-f3207948ba84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:58.772572 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-e0fddd58-9533-4e88-9295-f3207948ba84 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:58.773860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e0fddd58-9533-4e88-9295-f3207948ba84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:58.885234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:08:59.043172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4ea11-d2ad-4ff2-ad34-0b24bbd92b54 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:59.044813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0db4ea11-d2ad-4ff2-ad34-0b24bbd92b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:59.103151 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:59.266592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4ea11-d2ad-4ff2-ad34-0b24bbd92b54 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:59.421733 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5422eb-b0dd-4b12-b49c-9763b6e271f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:59.422348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5422eb-b0dd-4b12-b49c-9763b6e271f5 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:59.422620 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5422eb-b0dd-4b12-b49c-9763b6e271f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:59.422888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5422eb-b0dd-4b12-b49c-9763b6e271f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:59.424576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6e5422eb-b0dd-4b12-b49c-9763b6e271f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:59.432757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:59.532823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5422eb-b0dd-4b12-b49c-9763b6e271f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:59.534467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5422eb-b0dd-4b12-b49c-9763b6e271f5 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:59.556712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 718/1431] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:00.684493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0424a-bf09-479b-8d66-59d5c71d48b4 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:00.685038 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0424a-bf09-479b-8d66-59d5c71d48b4 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:00.685038 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0424a-bf09-479b-8d66-59d5c71d48b4 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:00.686424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:00.693773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1e444-bf76-4d15-9d06-2108693d65d0 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:00.694314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1e444-bf76-4d15-9d06-2108693d65d0 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:00.694314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1e444-bf76-4d15-9d06-2108693d65d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:00.785443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-bbe1e444-bf76-4d15-9d06-2108693d65d0 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:00.786684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-bbe1e444-bf76-4d15-9d06-2108693d65d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:00.868622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:00.903726 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:01.179383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0424a-bf09-479b-8d66-59d5c71d48b4 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:01.181065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-34f0424a-bf09-479b-8d66-59d5c71d48b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:01.202151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:01.413151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0424a-bf09-479b-8d66-59d5c71d48b4 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:01.579376 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:01.621609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e6fbd-759e-4c1b-9b8b-90a4a193f988 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:01.622229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e6fbd-759e-4c1b-9b8b-90a4a193f988 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:01.622505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e6fbd-759e-4c1b-9b8b-90a4a193f988 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:01.622772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e6fbd-759e-4c1b-9b8b-90a4a193f988 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:01.624474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-578e6fbd-759e-4c1b-9b8b-90a4a193f988 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:01.757903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e6fbd-759e-4c1b-9b8b-90a4a193f988 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:01.760617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e6fbd-759e-4c1b-9b8b-90a4a193f988 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:01.797882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 715/1434] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:02.927643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f0527-404d-4440-9dc9-b35557a4e186 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:02.928241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f0527-404d-4440-9dc9-b35557a4e186 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:02.928241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f0527-404d-4440-9dc9-b35557a4e186 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:02.929632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:09:02.937225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31d357-c49a-4b27-a57b-5c7e48723e94 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:02.937837 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31d357-c49a-4b27-a57b-5c7e48723e94 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:02.937837 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31d357-c49a-4b27-a57b-5c7e48723e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:03.030746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-9e31d357-c49a-4b27-a57b-5c7e48723e94 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:03.032209 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9e31d357-c49a-4b27-a57b-5c7e48723e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:03.091458 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:03.157121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:09:03.316565 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f0527-404d-4440-9dc9-b35557a4e186 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:03.318189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-899f0527-404d-4440-9dc9-b35557a4e186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:03.492504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f0527-404d-4440-9dc9-b35557a4e186 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:03.561478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:03.633412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47eb9d-d832-459c-a54f-d8618dbfbf3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:03.633944 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47eb9d-d832-459c-a54f-d8618dbfbf3d service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:03.634222 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47eb9d-d832-459c-a54f-d8618dbfbf3d service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:03.634618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47eb9d-d832-459c-a54f-d8618dbfbf3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:03.636503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-fa47eb9d-d832-459c-a54f-d8618dbfbf3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:03.770411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47eb9d-d832-459c-a54f-d8618dbfbf3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:03.772748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47eb9d-d832-459c-a54f-d8618dbfbf3d service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:03.794301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 721/1437] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:04.938713 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fd8aa-756d-41e0-b5ad-cc550f0c7716 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:04.939588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fd8aa-756d-41e0-b5ad-cc550f0c7716 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:04.939766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fd8aa-756d-41e0-b5ad-cc550f0c7716 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:04.942777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:09:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:04.952620 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51955d47-da81-42a9-9052-8cf5dd66cd64 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:04.953449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51955d47-da81-42a9-9052-8cf5dd66cd64 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:04.953449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51955d47-da81-42a9-9052-8cf5dd66cd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:05.047149 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-51955d47-da81-42a9-9052-8cf5dd66cd64 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:05.048542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-51955d47-da81-42a9-9052-8cf5dd66cd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:05.051045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:05.195280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:09:05.353881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fd8aa-756d-41e0-b5ad-cc550f0c7716 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:05.355677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e67fd8aa-756d-41e0-b5ad-cc550f0c7716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:05.531296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fd8aa-756d-41e0-b5ad-cc550f0c7716 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:05.562726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:05.667002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86c7d2-6e18-440a-ba0e-a21a9ecafbb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:05.667658 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86c7d2-6e18-440a-ba0e-a21a9ecafbb5 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:05.667939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86c7d2-6e18-440a-ba0e-a21a9ecafbb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:05.668210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86c7d2-6e18-440a-ba0e-a21a9ecafbb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:05.669942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7c86c7d2-6e18-440a-ba0e-a21a9ecafbb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:05.779613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86c7d2-6e18-440a-ba0e-a21a9ecafbb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:05.781672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86c7d2-6e18-440a-ba0e-a21a9ecafbb5 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:05.804214 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 718/1440] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:08.026483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcad0d-86dc-473b-9e36-5c2c051cc13e None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:08.027069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcad0d-86dc-473b-9e36-5c2c051cc13e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:08.027328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dcad0d-86dc-473b-9e36-5c2c051cc13e None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:08.028780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:09:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:08.037962 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431a679-11bd-4576-9dbe-dbb70e9bbbdd None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:08.038557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431a679-11bd-4576-9dbe-dbb70e9bbbdd None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:08.038557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431a679-11bd-4576-9dbe-dbb70e9bbbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:08.099789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:08.144471 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-2431a679-11bd-4576-9dbe-dbb70e9bbbdd None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:08.146160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2431a679-11bd-4576-9dbe-dbb70e9bbbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:08.271965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:08.569889 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcad0d-86dc-473b-9e36-5c2c051cc13e None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:08.571629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-20dcad0d-86dc-473b-9e36-5c2c051cc13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:08.701108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dcad0d-86dc-473b-9e36-5c2c051cc13e None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:08.740665 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:08.794800 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee82c5a-f7d3-4f57-a0cc-fb18f2c10a50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:08.795465 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee82c5a-f7d3-4f57-a0cc-fb18f2c10a50 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:08.795751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee82c5a-f7d3-4f57-a0cc-fb18f2c10a50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:08.796029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee82c5a-f7d3-4f57-a0cc-fb18f2c10a50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:08.797799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-aee82c5a-f7d3-4f57-a0cc-fb18f2c10a50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:08.953756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee82c5a-f7d3-4f57-a0cc-fb18f2c10a50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:08.955172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee82c5a-f7d3-4f57-a0cc-fb18f2c10a50 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:08.978844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 724/1443] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:11.688438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc5595-3134-46ad-83f2-1e4332468a53 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.689054 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc5595-3134-46ad-83f2-1e4332468a53 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.689054 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc5595-3134-46ad-83f2-1e4332468a53 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.690663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:11.697435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd817e2-3d97-4ee7-81ec-93b36860bce7 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.698017 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd817e2-3d97-4ee7-81ec-93b36860bce7 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.698017 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd817e2-3d97-4ee7-81ec-93b36860bce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.777516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-5dd817e2-3d97-4ee7-81ec-93b36860bce7 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:11.778767 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5dd817e2-3d97-4ee7-81ec-93b36860bce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:11.793759 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:11.920328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:09:12.209837 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc5595-3134-46ad-83f2-1e4332468a53 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:12.211483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d3dc5595-3134-46ad-83f2-1e4332468a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:12.346631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc5595-3134-46ad-83f2-1e4332468a53 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:12.363290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:12.476452 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1fd44-4d10-4506-8c59-bfe52fb9390c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:12.477510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1fd44-4d10-4506-8c59-bfe52fb9390c service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.478049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1fd44-4d10-4506-8c59-bfe52fb9390c service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.478685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1fd44-4d10-4506-8c59-bfe52fb9390c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.481636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-0ef1fd44-4d10-4506-8c59-bfe52fb9390c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:12.640902 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1fd44-4d10-4506-8c59-bfe52fb9390c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:12.643593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1fd44-4d10-4506-8c59-bfe52fb9390c service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:12.684432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 721/1446] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:15.517977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02886bd-6e80-4d52-aa41-edbd5ec5fca6 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:15.519458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02886bd-6e80-4d52-aa41-edbd5ec5fca6 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:15.519458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02886bd-6e80-4d52-aa41-edbd5ec5fca6 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:15.520938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:09:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:09:15.527433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47c3da-c910-4d82-a276-06ffa99d662b None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:15.528044 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47c3da-c910-4d82-a276-06ffa99d662b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:15.528044 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47c3da-c910-4d82-a276-06ffa99d662b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:15.586071 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:15.609605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-4a47c3da-c910-4d82-a276-06ffa99d662b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:15.610978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4a47c3da-c910-4d82-a276-06ffa99d662b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:15.737571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:09:16.049428 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02886bd-6e80-4d52-aa41-edbd5ec5fca6 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:16.051751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b02886bd-6e80-4d52-aa41-edbd5ec5fca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:16.212481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02886bd-6e80-4d52-aa41-edbd5ec5fca6 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:16.291613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:16.339966 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fcdef7-f3c9-4824-a143-23078a2f6de7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:16.340595 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcdef7-f3c9-4824-a143-23078a2f6de7 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:16.340871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcdef7-f3c9-4824-a143-23078a2f6de7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:16.341156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcdef7-f3c9-4824-a143-23078a2f6de7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:16.342987 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-15fcdef7-f3c9-4824-a143-23078a2f6de7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:16.486583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcdef7-f3c9-4824-a143-23078a2f6de7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:16.488836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fcdef7-f3c9-4824-a143-23078a2f6de7 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:16.511086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 727/1449] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:09:19.095890 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ebb7c-ca30-4793-a2d9-8ff034d1b113 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:19.097323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ebb7c-ca30-4793-a2d9-8ff034d1b113 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:19.097323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ebb7c-ca30-4793-a2d9-8ff034d1b113 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:19.098459 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:09:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:19.105255 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee5e9a-7252-447c-9f9d-e67a0c6e8f84 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:19.106157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee5e9a-7252-447c-9f9d-e67a0c6e8f84 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:19.106157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee5e9a-7252-447c-9f9d-e67a0c6e8f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:19.196410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-20ee5e9a-7252-447c-9f9d-e67a0c6e8f84 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:19.198390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-20ee5e9a-7252-447c-9f9d-e67a0c6e8f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:19.229074 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:19.340442 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:19.644881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ebb7c-ca30-4793-a2d9-8ff034d1b113 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:19.646495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-400ebb7c-ca30-4793-a2d9-8ff034d1b113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:19.791077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ebb7c-ca30-4793-a2d9-8ff034d1b113 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:19.825057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:19.947894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f4449-f98c-4335-9b31-0995f1f60cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:19.948964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f4449-f98c-4335-9b31-0995f1f60cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:19.949515 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f4449-f98c-4335-9b31-0995f1f60cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:19.950050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f4449-f98c-4335-9b31-0995f1f60cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:19.953769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2c6f4449-f98c-4335-9b31-0995f1f60cc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:20.120334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f4449-f98c-4335-9b31-0995f1f60cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:20.122302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f4449-f98c-4335-9b31-0995f1f60cc0 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:20.145223 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 724/1452] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:09:23.303317 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f17eb-9147-42ed-a93d-4c472a926bb5 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:23.303952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f17eb-9147-42ed-a93d-4c472a926bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:23.303952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f17eb-9147-42ed-a93d-4c472a926bb5 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:23.305262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:09:23.311549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b39c2-27ad-4328-851e-e63fc5d55261 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:23.312401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b39c2-27ad-4328-851e-e63fc5d55261 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:23.313257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b39c2-27ad-4328-851e-e63fc5d55261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:23.394599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:23.397949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-587b39c2-27ad-4328-851e-e63fc5d55261 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:23.400358 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-587b39c2-27ad-4328-851e-e63fc5d55261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:23.567229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:23.859972 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f17eb-9147-42ed-a93d-4c472a926bb5 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:23.861637 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-291f17eb-9147-42ed-a93d-4c472a926bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:24.033190 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f17eb-9147-42ed-a93d-4c472a926bb5 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:24.116344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:24.149138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679aa8e-1dd9-4275-9b41-5e3af9f56b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:24.149765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679aa8e-1dd9-4275-9b41-5e3af9f56b04 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:24.150042 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679aa8e-1dd9-4275-9b41-5e3af9f56b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:24.150316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679aa8e-1dd9-4275-9b41-5e3af9f56b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:24.152110 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-4679aa8e-1dd9-4275-9b41-5e3af9f56b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:24.290719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679aa8e-1dd9-4275-9b41-5e3af9f56b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:24.292345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679aa8e-1dd9-4275-9b41-5e3af9f56b04 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:24.313941 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 730/1455] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:09:27.119330 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ef06d-a8b1-4e58-aef5-bf2707990a7b None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.119950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ef06d-a8b1-4e58-aef5-bf2707990a7b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.120052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ef06d-a8b1-4e58-aef5-bf2707990a7b None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.121705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:27.127927 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef772f-f7b7-4663-9131-95769464ea36 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.128753 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef772f-f7b7-4663-9131-95769464ea36 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.128930 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef772f-f7b7-4663-9131-95769464ea36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.210597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8cef772f-f7b7-4663-9131-95769464ea36 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:27.212024 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8cef772f-f7b7-4663-9131-95769464ea36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:27.251660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:27.336458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:27.632500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ef06d-a8b1-4e58-aef5-bf2707990a7b None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:27.634200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f31ef06d-a8b1-4e58-aef5-bf2707990a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:27.756760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ef06d-a8b1-4e58-aef5-bf2707990a7b None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:27.791306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:27.878569 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c358b-5333-45de-8ee0-bfc1a630ede4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:27.879212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c358b-5333-45de-8ee0-bfc1a630ede4 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.879564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c358b-5333-45de-8ee0-bfc1a630ede4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.879898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c358b-5333-45de-8ee0-bfc1a630ede4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.881667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9f8c358b-5333-45de-8ee0-bfc1a630ede4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:28.016722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c358b-5333-45de-8ee0-bfc1a630ede4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:28.018249 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c358b-5333-45de-8ee0-bfc1a630ede4 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:28.042134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 727/1458] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:30.429144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f390563-31fa-483c-89ef-aa21ecc5cf66 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:30.429691 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f390563-31fa-483c-89ef-aa21ecc5cf66 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:30.429864 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f390563-31fa-483c-89ef-aa21ecc5cf66 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:30.431321 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:30.437659 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f26b7-2cc1-49b8-bdfb-35259431c940 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:30.438212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f26b7-2cc1-49b8-bdfb-35259431c940 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:30.438329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f26b7-2cc1-49b8-bdfb-35259431c940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:30.519490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-d26f26b7-2cc1-49b8-bdfb-35259431c940 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:30.520765 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d26f26b7-2cc1-49b8-bdfb-35259431c940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:30.530774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:30.639014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:30.928763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f390563-31fa-483c-89ef-aa21ecc5cf66 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:30.930367 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2f390563-31fa-483c-89ef-aa21ecc5cf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:31.053637 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f390563-31fa-483c-89ef-aa21ecc5cf66 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:31.130330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:31.154329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7a201-0983-40e6-98b0-81fe78bdc107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:31.155428 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7a201-0983-40e6-98b0-81fe78bdc107 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:31.155924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7a201-0983-40e6-98b0-81fe78bdc107 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:31.156411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7a201-0983-40e6-98b0-81fe78bdc107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:31.159938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-34a7a201-0983-40e6-98b0-81fe78bdc107 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:31.317565 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7a201-0983-40e6-98b0-81fe78bdc107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:31.319338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7a201-0983-40e6-98b0-81fe78bdc107 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:31.340392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 733/1461] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:34.112482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79d082-dbf6-4d8e-b818-f2fac9ec983b None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:34.113322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79d082-dbf6-4d8e-b818-f2fac9ec983b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:34.113322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79d082-dbf6-4d8e-b818-f2fac9ec983b None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:34.114577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:09:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:09:34.121925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31f1d0-0faf-4630-8822-672d724b5aff None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:34.122500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31f1d0-0faf-4630-8822-672d724b5aff None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:34.122871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31f1d0-0faf-4630-8822-672d724b5aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:34.217593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-9c31f1d0-0faf-4630-8822-672d724b5aff None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:34.219069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-9c31f1d0-0faf-4630-8822-672d724b5aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:34.281426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:34.375722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:34.683018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79d082-dbf6-4d8e-b818-f2fac9ec983b None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:34.684728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4d79d082-dbf6-4d8e-b818-f2fac9ec983b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:34.839625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79d082-dbf6-4d8e-b818-f2fac9ec983b None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:34.886952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:34.946814 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bee254-df7e-480f-ad3b-047b3d25f173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:34.947830 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bee254-df7e-480f-ad3b-047b3d25f173 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:34.948336 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bee254-df7e-480f-ad3b-047b3d25f173 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:34.948796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bee254-df7e-480f-ad3b-047b3d25f173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:34.952013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e8bee254-df7e-480f-ad3b-047b3d25f173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:35.140591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bee254-df7e-480f-ad3b-047b3d25f173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:35.142593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bee254-df7e-480f-ad3b-047b3d25f173 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:35.164948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 730/1464] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:09:37.544436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e88a5-4dc6-43b3-b3f4-a058245f8d8c None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:37.545020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e88a5-4dc6-43b3-b3f4-a058245f8d8c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:37.545225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e88a5-4dc6-43b3-b3f4-a058245f8d8c None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:37.547646 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:09:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:37.553747 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32939315-ffb6-44d1-a664-640cd757cc2a None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:37.554328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32939315-ffb6-44d1-a664-640cd757cc2a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:37.554328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32939315-ffb6-44d1-a664-640cd757cc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:37.630806 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-32939315-ffb6-44d1-a664-640cd757cc2a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:37.632005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-32939315-ffb6-44d1-a664-640cd757cc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:37.654138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:37.756052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:09:38.288689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e88a5-4dc6-43b3-b3f4-a058245f8d8c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:38.290449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f76e88a5-4dc6-43b3-b3f4-a058245f8d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:38.413005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e88a5-4dc6-43b3-b3f4-a058245f8d8c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:38.504958 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:38.515071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa6c53-26cd-444d-bcf9-8597fe639843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:38.515737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa6c53-26cd-444d-bcf9-8597fe639843 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:38.516015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa6c53-26cd-444d-bcf9-8597fe639843 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:38.516282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa6c53-26cd-444d-bcf9-8597fe639843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:38.518000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-82aa6c53-26cd-444d-bcf9-8597fe639843 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:38.645903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa6c53-26cd-444d-bcf9-8597fe639843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:38.647419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa6c53-26cd-444d-bcf9-8597fe639843 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:38.667918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 736/1467] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:09:41.024942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f43d18-8cbc-48e7-9eb7-7df87e0f157f None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:41.025538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f43d18-8cbc-48e7-9eb7-7df87e0f157f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:41.025538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f43d18-8cbc-48e7-9eb7-7df87e0f157f None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:41.027174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:09:41.034477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c6892-c49c-4341-bd10-59ec0c9fc4d6 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:41.035192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c6892-c49c-4341-bd10-59ec0c9fc4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:41.035192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c6892-c49c-4341-bd10-59ec0c9fc4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:41.121407 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ad3c6892-c49c-4341-bd10-59ec0c9fc4d6 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:41.122634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ad3c6892-c49c-4341-bd10-59ec0c9fc4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:41.361017 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:41.416959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:09:41.711442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f43d18-8cbc-48e7-9eb7-7df87e0f157f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:41.713093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-14f43d18-8cbc-48e7-9eb7-7df87e0f157f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:41.847963 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f43d18-8cbc-48e7-9eb7-7df87e0f157f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:41.910730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:41.949492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa439d-8574-4d8e-9375-52f58df2286f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:41.950546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa439d-8574-4d8e-9375-52f58df2286f service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:41.951046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa439d-8574-4d8e-9375-52f58df2286f service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:41.951578 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa439d-8574-4d8e-9375-52f58df2286f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:41.954846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b1fa439d-8574-4d8e-9375-52f58df2286f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:42.138405 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa439d-8574-4d8e-9375-52f58df2286f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:42.140032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa439d-8574-4d8e-9375-52f58df2286f service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:42.161906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 733/1470] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:09:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:09:51.414953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=22 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:52.159367 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=66 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:05.813258 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2c0eb-fc96-468c-b86b-6729ea2f03d2 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:05.814346 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2c0eb-fc96-468c-b86b-6729ea2f03d2 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:05.814346 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2c0eb-fc96-468c-b86b-6729ea2f03d2 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:05.815751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 738/1471] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:10:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:05.822012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ef0e9-92d4-411c-84f8-22ead1d2266a None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:05.822626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ef0e9-92d4-411c-84f8-22ead1d2266a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:05.822626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ef0e9-92d4-411c-84f8-22ead1d2266a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:05.901193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a28ef0e9-92d4-411c-84f8-22ead1d2266a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:05.902441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a28ef0e9-92d4-411c-84f8-22ead1d2266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:06.016928 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 734/1472] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:10:06.361387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ccfdd-419e-4ee7-82b1-3a0d5cd7c52a None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:06.361679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ccfdd-419e-4ee7-82b1-3a0d5cd7c52a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:06.361971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ccfdd-419e-4ee7-82b1-3a0d5cd7c52a None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:06.363361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 739/1473] 10.222.0.61 () {54 vars in 961 bytes} [Wed Sep 7 13:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:06.369345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492de70-1647-40fd-b3c5-60ef635e6033 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:06.369682 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492de70-1647-40fd-b3c5-60ef635e6033 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:06.370007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492de70-1647-40fd-b3c5-60ef635e6033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:06.475872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-8492de70-1647-40fd-b3c5-60ef635e6033 None None] MFA Rules not processed for user `b4f82adce75b44c6baf85b6b96dd0728`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:06.477864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-8492de70-1647-40fd-b3c5-60ef635e6033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:06.514426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:06.584889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 735/1474] 10.222.0.61 () {58 vars in 1058 bytes} [Wed Sep 7 13:10:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:10:06.594343 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ccfdd-419e-4ee7-82b1-3a0d5cd7c52a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:06.595982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-714ccfdd-419e-4ee7-82b1-3a0d5cd7c52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:06.722779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ccfdd-419e-4ee7-82b1-3a0d5cd7c52a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:06.826945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abed700-a932-49c6-8fbd-d635c908b2ca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f82adce75b44c6baf85b6b96dd0728', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:06.827955 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abed700-a932-49c6-8fbd-d635c908b2ca service glance] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:06.827955 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abed700-a932-49c6-8fbd-d635c908b2ca service glance] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:06.828346 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abed700-a932-49c6-8fbd-d635c908b2ca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:06.830372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-4abed700-a932-49c6-8fbd-d635c908b2ca service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:06.902346 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:06.972060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abed700-a932-49c6-8fbd-d635c908b2ca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:06.973543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abed700-a932-49c6-8fbd-d635c908b2ca service glance] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:06.994740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 740/1475] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:10:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:10:09.440977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ab129-acaa-402b-b50f-23d6b138fc8d None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:09.441553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ab129-acaa-402b-b50f-23d6b138fc8d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:09.441553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ab129-acaa-402b-b50f-23d6b138fc8d None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:09.442829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 736/1476] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:09.449153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e729b-0219-46cb-a4c1-62ea5ae9227a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:09.449945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e729b-0219-46cb-a4c1-62ea5ae9227a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:09.449945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e729b-0219-46cb-a4c1-62ea5ae9227a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:09.528686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-7b3e729b-0219-46cb-a4c1-62ea5ae9227a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:09.530144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7b3e729b-0219-46cb-a4c1-62ea5ae9227a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:09.645759 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 741/1477] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:10:09.927649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ab129-acaa-402b-b50f-23d6b138fc8d None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:09.929359 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c95ab129-acaa-402b-b50f-23d6b138fc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:10.115649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ab129-acaa-402b-b50f-23d6b138fc8d None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:10.208300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:10.268661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec9beb-c72f-4e47-bf5c-5ec6aec0c4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:10.269295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9beb-c72f-4e47-bf5c-5ec6aec0c4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:10.269573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9beb-c72f-4e47-bf5c-5ec6aec0c4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:10.269843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9beb-c72f-4e47-bf5c-5ec6aec0c4f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:10.270506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec9beb-c72f-4e47-bf5c-5ec6aec0c4f5 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:10.272408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: WARNING py.warnings [None req-efec9beb-c72f-4e47-bf5c-5ec6aec0c4f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:10:10.272408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: warnings.warn(msg) Sep 07 13:10:10.272408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: Sep 07 13:10:10.273597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec9beb-c72f-4e47-bf5c-5ec6aec0c4f5 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:10.296638 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 737/1478] 10.222.0.61 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:09 2022] GET /identity/v3/projects => generated 2553 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:10:12.734182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984a90c-529d-4613-8d8a-8c2dafcfd05b None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:12.734729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984a90c-529d-4613-8d8a-8c2dafcfd05b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:12.735052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984a90c-529d-4613-8d8a-8c2dafcfd05b None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:12.737701 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 742/1479] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:10:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:12.744462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d1390-1cf9-4b2e-823d-2451a19b7b83 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:12.745317 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d1390-1cf9-4b2e-823d-2451a19b7b83 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:12.745767 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d1390-1cf9-4b2e-823d-2451a19b7b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:12.839150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f92d1390-1cf9-4b2e-823d-2451a19b7b83 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:12.840808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f92d1390-1cf9-4b2e-823d-2451a19b7b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:12.968567 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 738/1480] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:10:13.280292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984a90c-529d-4613-8d8a-8c2dafcfd05b None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:13.282699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7984a90c-529d-4613-8d8a-8c2dafcfd05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:13.293403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:13.451365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984a90c-529d-4613-8d8a-8c2dafcfd05b None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:13.563914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:13.603599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc32fa-8ebb-426e-a40b-0bbe592eac38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:13.604226 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc32fa-8ebb-426e-a40b-0bbe592eac38 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:13.604507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc32fa-8ebb-426e-a40b-0bbe592eac38 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:13.604780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc32fa-8ebb-426e-a40b-0bbe592eac38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:13.606562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0dcc32fa-8ebb-426e-a40b-0bbe592eac38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:13.714934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc32fa-8ebb-426e-a40b-0bbe592eac38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:13.716938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc32fa-8ebb-426e-a40b-0bbe592eac38 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:13.738389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 743/1481] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:10:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:10:16.258085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3ff44-1de9-4734-94ed-ffe1ad9ccedb None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:16.258718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3ff44-1de9-4734-94ed-ffe1ad9ccedb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:16.258718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3ff44-1de9-4734-94ed-ffe1ad9ccedb None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:16.260042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 739/1482] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:16.266801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbe467-fbf6-4149-94df-70c1f956de7d None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:16.267349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbe467-fbf6-4149-94df-70c1f956de7d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:16.267612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbe467-fbf6-4149-94df-70c1f956de7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:16.348764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f0fbe467-fbf6-4149-94df-70c1f956de7d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:16.349912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f0fbe467-fbf6-4149-94df-70c1f956de7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:16.403665 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:16.478604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 744/1483] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:10:16.788036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3ff44-1de9-4734-94ed-ffe1ad9ccedb None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:16.789603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b7c3ff44-1de9-4734-94ed-ffe1ad9ccedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:16.816612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:16.923892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3ff44-1de9-4734-94ed-ffe1ad9ccedb None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:17.016634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22085145-20e7-4008-b285-a5fa3192e6b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:17.017248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22085145-20e7-4008-b285-a5fa3192e6b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:17.017524 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22085145-20e7-4008-b285-a5fa3192e6b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:17.017795 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22085145-20e7-4008-b285-a5fa3192e6b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:17.019532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-22085145-20e7-4008-b285-a5fa3192e6b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:17.106216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:17.147142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22085145-20e7-4008-b285-a5fa3192e6b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:17.148769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22085145-20e7-4008-b285-a5fa3192e6b7 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:17.170074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 740/1484] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:10:19.519601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdcbb3-6829-4a47-a633-e25038265d90 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:19.520151 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdcbb3-6829-4a47-a633-e25038265d90 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:19.520151 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdcbb3-6829-4a47-a633-e25038265d90 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:19.521399 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 745/1485] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:19.527604 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67677604-f496-4072-b309-98cfa08f6beb None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:19.528153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67677604-f496-4072-b309-98cfa08f6beb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:19.528371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67677604-f496-4072-b309-98cfa08f6beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:19.614455 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-67677604-f496-4072-b309-98cfa08f6beb None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:19.615775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-67677604-f496-4072-b309-98cfa08f6beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:19.729589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 741/1486] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:10:20.030587 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdcbb3-6829-4a47-a633-e25038265d90 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:20.033784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8ffdcbb3-6829-4a47-a633-e25038265d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:20.159298 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdcbb3-6829-4a47-a633-e25038265d90 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:20.249597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:20.252387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1172d89-e93d-4b19-bade-c199cd75304d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:20.253152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1172d89-e93d-4b19-bade-c199cd75304d service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:20.253700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1172d89-e93d-4b19-bade-c199cd75304d service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:20.254208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1172d89-e93d-4b19-bade-c199cd75304d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:20.256424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f1172d89-e93d-4b19-bade-c199cd75304d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:20.420343 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1172d89-e93d-4b19-bade-c199cd75304d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:20.421993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1172d89-e93d-4b19-bade-c199cd75304d service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:20.445184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 746/1487] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:10:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:10:22.908919 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a955c56-7393-4ed1-b167-6ab901ebab31 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:22.909474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a955c56-7393-4ed1-b167-6ab901ebab31 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:22.909781 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a955c56-7393-4ed1-b167-6ab901ebab31 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:22.911824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 742/1488] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:10:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:22.918201 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e939c-93e2-4aa7-8c84-5a804b36295d None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:22.918791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e939c-93e2-4aa7-8c84-5a804b36295d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:22.918791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e939c-93e2-4aa7-8c84-5a804b36295d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:22.996193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-7c9e939c-93e2-4aa7-8c84-5a804b36295d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:22.997431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7c9e939c-93e2-4aa7-8c84-5a804b36295d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:23.057326 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:23.106877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 747/1489] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:10:23.404059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a955c56-7393-4ed1-b167-6ab901ebab31 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:23.405640 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5a955c56-7393-4ed1-b167-6ab901ebab31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:23.446856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:23.528276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a955c56-7393-4ed1-b167-6ab901ebab31 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:23.617936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3e561-59c9-4ca1-9bf9-624ff573a82d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:23.618667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e561-59c9-4ca1-9bf9-624ff573a82d service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:23.619011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e561-59c9-4ca1-9bf9-624ff573a82d service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:23.619456 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e561-59c9-4ca1-9bf9-624ff573a82d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:23.621538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-afc3e561-59c9-4ca1-9bf9-624ff573a82d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:23.724038 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:23.759070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3e561-59c9-4ca1-9bf9-624ff573a82d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:23.760552 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3e561-59c9-4ca1-9bf9-624ff573a82d service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:23.786585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 743/1490] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:10:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:10:26.620580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad724d-9990-4786-a4bf-ca4b1df277f0 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:26.621182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad724d-9990-4786-a4bf-ca4b1df277f0 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:26.621182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad724d-9990-4786-a4bf-ca4b1df277f0 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:26.622478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 748/1491] 10.222.0.61 () {54 vars in 927 bytes} [Wed Sep 7 13:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:26.629425 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1e489-e929-4476-8cf0-c0ad63cff3c9 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:26.630032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1e489-e929-4476-8cf0-c0ad63cff3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:26.630032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1e489-e929-4476-8cf0-c0ad63cff3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:26.713155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-4cd1e489-e929-4476-8cf0-c0ad63cff3c9 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:26.714329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4cd1e489-e929-4476-8cf0-c0ad63cff3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:26.827714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 744/1492] 10.222.0.61 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:10:27.144985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad724d-9990-4786-a4bf-ca4b1df277f0 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:27.147686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-97ad724d-9990-4786-a4bf-ca4b1df277f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:27.286446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad724d-9990-4786-a4bf-ca4b1df277f0 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:27.383962 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c3b9c-681a-4d99-96ac-5961a56864ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:27.384840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c3b9c-681a-4d99-96ac-5961a56864ae service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:27.385248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c3b9c-681a-4d99-96ac-5961a56864ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:27.385635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c3b9c-681a-4d99-96ac-5961a56864ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:27.388328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f70c3b9c-681a-4d99-96ac-5961a56864ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:27.399936 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:27.525411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c3b9c-681a-4d99-96ac-5961a56864ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:27.526937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c3b9c-681a-4d99-96ac-5961a56864ae service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:27.549234 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 749/1493] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:10:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:10:36.825830 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=82 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:37.546115 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=50 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:16.742559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874c7ad-30d8-4812-92fd-51b80af94077 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:16.743328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874c7ad-30d8-4812-92fd-51b80af94077 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:16.743328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874c7ad-30d8-4812-92fd-51b80af94077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:16.862774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f874c7ad-30d8-4812-92fd-51b80af94077 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:12:16.864068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f874c7ad-30d8-4812-92fd-51b80af94077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:16.995902 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 745/1494] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:12:17.005509 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e863a7-75c6-4c9d-bd3d-12cf73b07d34 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:17.006238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e863a7-75c6-4c9d-bd3d-12cf73b07d34 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:17.006238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e863a7-75c6-4c9d-bd3d-12cf73b07d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:17.030464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f3e863a7-75c6-4c9d-bd3d-12cf73b07d34 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:12:17.031752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f3e863a7-75c6-4c9d-bd3d-12cf73b07d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:17.041509 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 750/1495] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:12:17.047091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874c7ad-30d8-4812-92fd-51b80af94077 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:17.065067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874c7ad-30d8-4812-92fd-51b80af94077 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:17.073866 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cc422-c7e9-4379-b371-b56acdcb45c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:17.074936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cc422-c7e9-4379-b371-b56acdcb45c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:17.075424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cc422-c7e9-4379-b371-b56acdcb45c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:17.075928 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cc422-c7e9-4379-b371-b56acdcb45c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:17.077144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cc422-c7e9-4379-b371-b56acdcb45c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:12:17.077678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cc422-c7e9-4379-b371-b56acdcb45c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:17.079655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: WARNING py.warnings [None req-223cc422-c7e9-4379-b371-b56acdcb45c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:12:17.079655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: warnings.warn(msg) Sep 07 13:12:17.079655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: Sep 07 13:12:17.081389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cc422-c7e9-4379-b371-b56acdcb45c8 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:17.104372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 746/1496] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:12:17.115010 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e863a7-75c6-4c9d-bd3d-12cf73b07d34 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:17.125838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e863a7-75c6-4c9d-bd3d-12cf73b07d34 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:17.131294 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03bc92-17cb-449f-a9a8-8e29635870d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:17.131997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03bc92-17cb-449f-a9a8-8e29635870d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:17.132412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03bc92-17cb-449f-a9a8-8e29635870d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:17.132725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03bc92-17cb-449f-a9a8-8e29635870d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:17.133445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03bc92-17cb-449f-a9a8-8e29635870d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:17.135418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: WARNING py.warnings [None req-6a03bc92-17cb-449f-a9a8-8e29635870d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:12:17.135418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: warnings.warn(msg) Sep 07 13:12:17.135418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: Sep 07 13:12:17.137570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03bc92-17cb-449f-a9a8-8e29635870d2 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:17.189254 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 751/1497] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:12:17 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:12:17.196114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cc422-c7e9-4379-b371-b56acdcb45c8 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:17.215315 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cc422-c7e9-4379-b371-b56acdcb45c8 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:17.283590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:17.291516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf04da-a703-4cc3-9048-e430ac7790c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:17.292115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf04da-a703-4cc3-9048-e430ac7790c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:17.292522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf04da-a703-4cc3-9048-e430ac7790c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:17.292683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf04da-a703-4cc3-9048-e430ac7790c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:17.293435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf04da-a703-4cc3-9048-e430ac7790c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:17.294538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: WARNING py.warnings [None req-8fbf04da-a703-4cc3-9048-e430ac7790c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:12:17.294538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: warnings.warn(msg) Sep 07 13:12:17.294538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: Sep 07 13:12:17.295808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf04da-a703-4cc3-9048-e430ac7790c9 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:17.320997 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 747/1498] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:12:17 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:12:17.326177 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03bc92-17cb-449f-a9a8-8e29635870d2 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:17.337855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03bc92-17cb-449f-a9a8-8e29635870d2 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:17.342739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01c5cb-9103-4bf0-9e8d-325f92174556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:17.343392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01c5cb-9103-4bf0-9e8d-325f92174556 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:17.343670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01c5cb-9103-4bf0-9e8d-325f92174556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:17.344096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01c5cb-9103-4bf0-9e8d-325f92174556 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:17.344762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01c5cb-9103-4bf0-9e8d-325f92174556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:17.345809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: WARNING py.warnings [None req-4c01c5cb-9103-4bf0-9e8d-325f92174556 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:12:17.345809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: warnings.warn(msg) Sep 07 13:12:17.345809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: Sep 07 13:12:17.346705 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01c5cb-9103-4bf0-9e8d-325f92174556 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:17.362043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 752/1499] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:12:17.366964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf04da-a703-4cc3-9048-e430ac7790c9 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:17.387458 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf04da-a703-4cc3-9048-e430ac7790c9 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:17.395735 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a0a52-3ef2-400a-b711-be671cb18e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:17.396516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a0a52-3ef2-400a-b711-be671cb18e18 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:17.396994 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a0a52-3ef2-400a-b711-be671cb18e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:17.397469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a0a52-3ef2-400a-b711-be671cb18e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:17.398630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a0a52-3ef2-400a-b711-be671cb18e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:17.400453 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: WARNING py.warnings [None req-658a0a52-3ef2-400a-b711-be671cb18e18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:12:17.400453 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: warnings.warn(msg) Sep 07 13:12:17.400453 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: Sep 07 13:12:17.402165 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a0a52-3ef2-400a-b711-be671cb18e18 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:17.429295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 748/1500] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:12:17.434977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01c5cb-9103-4bf0-9e8d-325f92174556 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:17.452085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01c5cb-9103-4bf0-9e8d-325f92174556 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:17.457379 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e562a-07b4-4c49-82ba-9f8b62103fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:17.458063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e562a-07b4-4c49-82ba-9f8b62103fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:17.458341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e562a-07b4-4c49-82ba-9f8b62103fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:17.458613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e562a-07b4-4c49-82ba-9f8b62103fd4 admin admin] PATH_INFO: `/v3/projects/0b6c82a1e7f94e499e627ee07e7fc971/users/74066567ec2045ed931daa100016604c/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:17.475760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e562a-07b4-4c49-82ba-9f8b62103fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6c82a1e7f94e499e627ee07e7fc971, user_id=74066567ec2045ed931daa100016604c, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:17.476828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: WARNING py.warnings [None req-1e6e562a-07b4-4c49-82ba-9f8b62103fd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:12:17.476828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: warnings.warn(msg) Sep 07 13:12:17.476828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: Sep 07 13:12:17.478833 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e562a-07b4-4c49-82ba-9f8b62103fd4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:17.495285 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 753/1501] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:12:17 2022] PUT /identity/v3/projects/0b6c82a1e7f94e499e627ee07e7fc971/users/74066567ec2045ed931daa100016604c/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:12:17.507833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd178b32-42c1-4a14-9e7d-80ed0ee91c22 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:17.508108 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd178b32-42c1-4a14-9e7d-80ed0ee91c22 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:17.508403 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd178b32-42c1-4a14-9e7d-80ed0ee91c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:17.540809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-dd178b32-42c1-4a14-9e7d-80ed0ee91c22 None None] MFA Rules not processed for user `74066567ec2045ed931daa100016604c`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:12:17.542182 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-dd178b32-42c1-4a14-9e7d-80ed0ee91c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:17.619203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 749/1502] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:12:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:12:17.634613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3d4eb-f973-4037-9968-38a110ae087d None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:17.635125 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3d4eb-f973-4037-9968-38a110ae087d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:17.636216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3d4eb-f973-4037-9968-38a110ae087d None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:17.638764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 754/1503] 10.222.0.61 () {54 vars in 964 bytes} [Wed Sep 7 13:12:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:12:17.645422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eaaaf-658e-4ac0-b881-78bb5e7d59cb None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:17.645804 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eaaaf-658e-4ac0-b881-78bb5e7d59cb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:17.646113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eaaaf-658e-4ac0-b881-78bb5e7d59cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:17.727397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-df9eaaaf-658e-4ac0-b881-78bb5e7d59cb None None] MFA Rules not processed for user `8356f60a0f684484abc8f8d6fb6a7be6`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:12:17.728753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-df9eaaaf-658e-4ac0-b881-78bb5e7d59cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:17.833953 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 750/1504] 10.222.0.61 () {58 vars in 1061 bytes} [Wed Sep 7 13:12:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:12:17.850321 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3d4eb-f973-4037-9968-38a110ae087d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:17.862563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3d4eb-f973-4037-9968-38a110ae087d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:17.867507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34a36f-c571-481e-a19c-44d8a1f78dbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:17.868153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34a36f-c571-481e-a19c-44d8a1f78dbe service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:17.868697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34a36f-c571-481e-a19c-44d8a1f78dbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:17.868967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34a36f-c571-481e-a19c-44d8a1f78dbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:17.879630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34a36f-c571-481e-a19c-44d8a1f78dbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:17.880963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34a36f-c571-481e-a19c-44d8a1f78dbe service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:17.891956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 755/1505] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:12:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:12:17.947269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf40a76-a981-4907-a685-534269b44016 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:17.947803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf40a76-a981-4907-a685-534269b44016 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:17.948661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf40a76-a981-4907-a685-534269b44016 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:17.950236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 751/1506] 10.222.0.61 () {52 vars in 830 bytes} [Wed Sep 7 13:12:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:12:17.955655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34a36f-c571-481e-a19c-44d8a1f78dbe service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:18.043749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34a36f-c571-481e-a19c-44d8a1f78dbe service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:18.051767 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:18.052874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:18.053374 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:18.053824 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] PATH_INFO: `/v3/users/74066567ec2045ed931daa100016604c` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:18.056105 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=74066567ec2045ed931daa100016604c)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:18.058654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: WARNING py.warnings [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:12:18.058654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: warnings.warn(msg) Sep 07 13:12:18.058654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: Sep 07 13:12:18.061192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:18.154331 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74066567ec2045ed931daa100016604c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:18.166241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74066567ec2045ed931daa100016604c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:18.167056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74066567ec2045ed931daa100016604c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:18.167795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74066567ec2045ed931daa100016604c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:18.199820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74066567ec2045ed931daa100016604c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:18.212676 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:18.227507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 756/1507] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:12:17 2022] DELETE /identity/v3/users/74066567ec2045ed931daa100016604c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:12:18.235234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c548756-96db-4352-a8d3-84650d835d30 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:18.235748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c548756-96db-4352-a8d3-84650d835d30 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:18.236301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c548756-96db-4352-a8d3-84650d835d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:18.271271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-1c548756-96db-4352-a8d3-84650d835d30 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:12:18.273381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1c548756-96db-4352-a8d3-84650d835d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:18.293280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:18.381522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 752/1508] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:12:18.395228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:18.396833 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:18.493180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828e3c2-bba9-45f3-a25d-f7578dbbde0a admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:18.497989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877cc36-aafe-4bcf-8e46-e6d1c340220b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:18.498736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877cc36-aafe-4bcf-8e46-e6d1c340220b service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:18.499058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877cc36-aafe-4bcf-8e46-e6d1c340220b service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:18.499456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877cc36-aafe-4bcf-8e46-e6d1c340220b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:18.511763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877cc36-aafe-4bcf-8e46-e6d1c340220b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:18.513248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877cc36-aafe-4bcf-8e46-e6d1c340220b service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:18.526929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 757/1509] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:12:19.029383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c548756-96db-4352-a8d3-84650d835d30 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:19.031007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1c548756-96db-4352-a8d3-84650d835d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:19.046345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c548756-96db-4352-a8d3-84650d835d30 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:19.051675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:19.052388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:19.052776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:19.053151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0 admin admin] PATH_INFO: `/v3/projects/0b6c82a1e7f94e499e627ee07e7fc971` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:19.054236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6c82a1e7f94e499e627ee07e7fc971)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:19.055329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: WARNING py.warnings [None req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:12:19.055329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: warnings.warn(msg) Sep 07 13:12:19.055329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: Sep 07 13:12:19.056561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:19.146072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6c82a1e7f94e499e627ee07e7fc971', 'request_id': 'req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:19.163694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6c82a1e7f94e499e627ee07e7fc971', 'request_id': 'req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:19.172581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6c82a1e7f94e499e627ee07e7fc971', 'request_id': 'req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:19.192553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0 admin admin] The token cache is being invalidate because project 0b6c82a1e7f94e499e627ee07e7fc971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:12:19.192998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:19.193911 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1cb7cdc2-6eac-4e2b-91c9-79437ec083c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:12:19.196570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 753/1510] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:12:19 2022] DELETE /identity/v3/projects/0b6c82a1e7f94e499e627ee07e7fc971 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:12:28.520121 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=38 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:29.188373 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:01.164309 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402663fd-42fc-4916-8b7d-1d7cd13b868d None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:01.164929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402663fd-42fc-4916-8b7d-1d7cd13b868d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:01.164929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402663fd-42fc-4916-8b7d-1d7cd13b868d None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:01.166458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:14:01.171864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698575e-40ff-4ded-afe9-073bceef222e None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:01.172399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698575e-40ff-4ded-afe9-073bceef222e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:01.172512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698575e-40ff-4ded-afe9-073bceef222e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:01.195623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-e698575e-40ff-4ded-afe9-073bceef222e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:01.196867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e698575e-40ff-4ded-afe9-073bceef222e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:01.384978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:14:01.548352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402663fd-42fc-4916-8b7d-1d7cd13b868d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:01.550945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-402663fd-42fc-4916-8b7d-1d7cd13b868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:01.673721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402663fd-42fc-4916-8b7d-1d7cd13b868d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:01.680479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80c50d-4ea1-459e-b0d7-bc2ee69df3da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:01.681180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80c50d-4ea1-459e-b0d7-bc2ee69df3da service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:01.681461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80c50d-4ea1-459e-b0d7-bc2ee69df3da service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:01.681736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80c50d-4ea1-459e-b0d7-bc2ee69df3da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:01.690915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80c50d-4ea1-459e-b0d7-bc2ee69df3da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:01.692572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80c50d-4ea1-459e-b0d7-bc2ee69df3da service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:01.703262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 759/1513] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:14:02.841422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d53a5-49c0-41f2-99de-6dafcc4c3b85 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:02.841991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d53a5-49c0-41f2-99de-6dafcc4c3b85 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:02.841991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d53a5-49c0-41f2-99de-6dafcc4c3b85 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:02.843437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:14:02.850395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d99ac9-f4bd-44f4-bfb6-8feee77d39ce None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:02.851155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d99ac9-f4bd-44f4-bfb6-8feee77d39ce None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:02.851155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d99ac9-f4bd-44f4-bfb6-8feee77d39ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:02.872403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-b2d99ac9-f4bd-44f4-bfb6-8feee77d39ce None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:02.873641 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b2d99ac9-f4bd-44f4-bfb6-8feee77d39ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:02.882377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:14:03.038212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d53a5-49c0-41f2-99de-6dafcc4c3b85 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:03.053731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d53a5-49c0-41f2-99de-6dafcc4c3b85 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:03.059423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9f4d4-e17a-4f59-83c8-55415ea704a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:03.060036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9f4d4-e17a-4f59-83c8-55415ea704a3 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:03.060315 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9f4d4-e17a-4f59-83c8-55415ea704a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:03.060587 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9f4d4-e17a-4f59-83c8-55415ea704a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:03.070351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9f4d4-e17a-4f59-83c8-55415ea704a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:03.072019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9f4d4-e17a-4f59-83c8-55415ea704a3 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:03.082479 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 756/1516] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:14:05.910896 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fd27b-06fe-4e22-9312-eb49942aedc5 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:05.911572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fd27b-06fe-4e22-9312-eb49942aedc5 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:05.911572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fd27b-06fe-4e22-9312-eb49942aedc5 None None] PATH_INFO: `/v3` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:05.912848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 761/1517] 10.222.0.72 () {54 vars in 925 bytes} [Wed Sep 7 13:14:05 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:14:05.921643 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830cbea-4155-4984-999a-3ac5c959a92b None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:05.922221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830cbea-4155-4984-999a-3ac5c959a92b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:05.922358 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830cbea-4155-4984-999a-3ac5c959a92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:05.977560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-d830cbea-4155-4984-999a-3ac5c959a92b None None] MFA Rules not processed for user `9e283e62741f40439d02e545a212807a`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:05.978799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d830cbea-4155-4984-999a-3ac5c959a92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:06.059035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 757/1518] 10.222.0.72 () {58 vars in 1015 bytes} [Wed Sep 7 13:14:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:14:12.867069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=49 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:14.639946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e9e01-9c70-4ab7-a411-77b7b18277d4 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:14.640616 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e9e01-9c70-4ab7-a411-77b7b18277d4 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:14.640616 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e9e01-9c70-4ab7-a411-77b7b18277d4 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:14.641904 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 762/1519] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:14:14.652260 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fa6ab-3997-4c2f-813c-70db58e6dc0e None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:14.653097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fa6ab-3997-4c2f-813c-70db58e6dc0e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:14.653097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fa6ab-3997-4c2f-813c-70db58e6dc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:14.674892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-698fa6ab-3997-4c2f-813c-70db58e6dc0e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:14.676232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-698fa6ab-3997-4c2f-813c-70db58e6dc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:14.685239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 758/1520] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:14:14.845797 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e9e01-9c70-4ab7-a411-77b7b18277d4 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:14.861551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e9e01-9c70-4ab7-a411-77b7b18277d4 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:14.867322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204a77a-53f2-4dfe-867c-f043355ef002 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:14.867944 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204a77a-53f2-4dfe-867c-f043355ef002 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:14.868228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204a77a-53f2-4dfe-867c-f043355ef002 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:14.868499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204a77a-53f2-4dfe-867c-f043355ef002 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:14.877746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204a77a-53f2-4dfe-867c-f043355ef002 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:14.879399 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204a77a-53f2-4dfe-867c-f043355ef002 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:14.889734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 763/1521] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:14:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:14:16.038238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1363a-3e60-42bc-ab5f-8e2848c08453 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:16.039086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1363a-3e60-42bc-ab5f-8e2848c08453 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:16.039281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1363a-3e60-42bc-ab5f-8e2848c08453 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:16.040799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:14:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:14:16.047796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553ac9a-6c20-40af-86e4-6c8e860aca89 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:16.048369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553ac9a-6c20-40af-86e4-6c8e860aca89 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:16.048369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553ac9a-6c20-40af-86e4-6c8e860aca89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:16.070068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-2553ac9a-6c20-40af-86e4-6c8e860aca89 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:16.072275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2553ac9a-6c20-40af-86e4-6c8e860aca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:16.086515 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:14:16.247585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1363a-3e60-42bc-ab5f-8e2848c08453 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:16.264013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1363a-3e60-42bc-ab5f-8e2848c08453 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:16.269683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292e298-ce14-408a-8ba6-d3d2af76224a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:16.270312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292e298-ce14-408a-8ba6-d3d2af76224a service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:16.270654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292e298-ce14-408a-8ba6-d3d2af76224a service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:16.271475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292e298-ce14-408a-8ba6-d3d2af76224a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:16.280849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292e298-ce14-408a-8ba6-d3d2af76224a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:16.282529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292e298-ce14-408a-8ba6-d3d2af76224a service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:16.293836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 760/1524] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:14:18.336689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a95c53-1f49-4b86-95c4-9a69a8c061ec None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:18.337231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a95c53-1f49-4b86-95c4-9a69a8c061ec None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:18.337477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a95c53-1f49-4b86-95c4-9a69a8c061ec None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:18.339860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 765/1525] 10.222.0.61 () {54 vars in 961 bytes} [Wed Sep 7 13:14:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:14:18.349646 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773c3ea-7062-4117-bfdc-ad6665402bcc None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:18.350229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773c3ea-7062-4117-bfdc-ad6665402bcc None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:18.350644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773c3ea-7062-4117-bfdc-ad6665402bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:18.372873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-d773c3ea-7062-4117-bfdc-ad6665402bcc None None] MFA Rules not processed for user `9e283e62741f40439d02e545a212807a`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:18.374124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d773c3ea-7062-4117-bfdc-ad6665402bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:18.383182 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 761/1526] 10.222.0.61 () {58 vars in 1058 bytes} [Wed Sep 7 13:14:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:14:18.391558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a95c53-1f49-4b86-95c4-9a69a8c061ec None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:18.405459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a95c53-1f49-4b86-95c4-9a69a8c061ec None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:18.410858 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdde28d-1d9c-4be1-aacf-488f9f6f952c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:18.412043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdde28d-1d9c-4be1-aacf-488f9f6f952c service placement] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:18.412335 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdde28d-1d9c-4be1-aacf-488f9f6f952c service placement] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:18.412603 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdde28d-1d9c-4be1-aacf-488f9f6f952c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:18.422087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdde28d-1d9c-4be1-aacf-488f9f6f952c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:18.423743 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdde28d-1d9c-4be1-aacf-488f9f6f952c service placement] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:18.435431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 766/1527] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:14:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:14:27.886342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bc918-181b-43e0-97aa-b783d28f8fba None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:27.887052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bc918-181b-43e0-97aa-b783d28f8fba None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:27.887052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bc918-181b-43e0-97aa-b783d28f8fba None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:27.888323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:14:27.895925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45314a3-94a8-46ff-9a48-69d1401c4688 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:27.896818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45314a3-94a8-46ff-9a48-69d1401c4688 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:27.896818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45314a3-94a8-46ff-9a48-69d1401c4688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:27.918005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-d45314a3-94a8-46ff-9a48-69d1401c4688 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:27.919336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d45314a3-94a8-46ff-9a48-69d1401c4688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:27.927569 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:14:28.088064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bc918-181b-43e0-97aa-b783d28f8fba None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:28.107798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bc918-181b-43e0-97aa-b783d28f8fba None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:28.113655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc8f99-852a-4ebf-b878-b7167c49d21d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:28.114280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc8f99-852a-4ebf-b878-b7167c49d21d service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:28.114559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc8f99-852a-4ebf-b878-b7167c49d21d service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:28.115081 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc8f99-852a-4ebf-b878-b7167c49d21d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:28.131098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc8f99-852a-4ebf-b878-b7167c49d21d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:28.133439 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc8f99-852a-4ebf-b878-b7167c49d21d service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:28.145945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 763/1530] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:14:29.441096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e165226-17e0-43b2-aab3-d9758cbf0e32 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:29.442321 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e165226-17e0-43b2-aab3-d9758cbf0e32 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:29.442321 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e165226-17e0-43b2-aab3-d9758cbf0e32 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:29.445592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:14:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:14:29.451611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9f4de-a5c7-45ec-bde9-991caf05fefd None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:29.452163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9f4de-a5c7-45ec-bde9-991caf05fefd None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:29.452163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9f4de-a5c7-45ec-bde9-991caf05fefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:29.461734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:29.477957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-16e9f4de-a5c7-45ec-bde9-991caf05fefd None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:29.480138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-16e9f4de-a5c7-45ec-bde9-991caf05fefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:29.495270 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:14:29.676281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e165226-17e0-43b2-aab3-d9758cbf0e32 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:29.693981 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e165226-17e0-43b2-aab3-d9758cbf0e32 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:29.701297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f5e3d-9e3d-4f8d-a69b-36d3042037eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:29.701922 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f5e3d-9e3d-4f8d-a69b-36d3042037eb service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:29.702205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f5e3d-9e3d-4f8d-a69b-36d3042037eb service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:29.702477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f5e3d-9e3d-4f8d-a69b-36d3042037eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:29.715699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f5e3d-9e3d-4f8d-a69b-36d3042037eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:29.717405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f5e3d-9e3d-4f8d-a69b-36d3042037eb service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:29.728938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 769/1533] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:14:34.343178 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8056c2-71c8-4ea4-b184-4f4d177add22 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:34.343774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8056c2-71c8-4ea4-b184-4f4d177add22 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:34.343892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8056c2-71c8-4ea4-b184-4f4d177add22 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:34.346185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:14:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:14:34.355486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dca972-8d8b-4a0c-a22f-5afabd79c243 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:34.356089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dca972-8d8b-4a0c-a22f-5afabd79c243 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:34.356089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dca972-8d8b-4a0c-a22f-5afabd79c243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:34.376957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-95dca972-8d8b-4a0c-a22f-5afabd79c243 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:34.378174 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-95dca972-8d8b-4a0c-a22f-5afabd79c243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:34.386329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:14:34.704563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8056c2-71c8-4ea4-b184-4f4d177add22 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:34.706407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-af8056c2-71c8-4ea4-b184-4f4d177add22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:34.777730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8056c2-71c8-4ea4-b184-4f4d177add22 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:34.782170 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c805c4-0076-4925-9122-fb044b4d2599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:34.782875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c805c4-0076-4925-9122-fb044b4d2599 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:34.783240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c805c4-0076-4925-9122-fb044b4d2599 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:34.783564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c805c4-0076-4925-9122-fb044b4d2599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:34.794722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c805c4-0076-4925-9122-fb044b4d2599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:34.796252 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c805c4-0076-4925-9122-fb044b4d2599 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:34.808239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 766/1536] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:14:37.195703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5ae39-d38d-4dd8-9269-0121685e306a None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:37.196486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5ae39-d38d-4dd8-9269-0121685e306a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:37.196486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5ae39-d38d-4dd8-9269-0121685e306a None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:37.198131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:14:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:14:37.204443 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571a8f2-528e-4e15-ac8b-9f3bd4d11f5a None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:37.205081 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571a8f2-528e-4e15-ac8b-9f3bd4d11f5a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:37.205081 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571a8f2-528e-4e15-ac8b-9f3bd4d11f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:37.226388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-6571a8f2-528e-4e15-ac8b-9f3bd4d11f5a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:37.227803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-6571a8f2-528e-4e15-ac8b-9f3bd4d11f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:37.236784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:14:37.537671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5ae39-d38d-4dd8-9269-0121685e306a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:37.556867 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5ae39-d38d-4dd8-9269-0121685e306a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:37.563275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4eb0b-838e-4d6f-9907-60a21c779c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:37.564323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4eb0b-838e-4d6f-9907-60a21c779c7c service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:37.564806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4eb0b-838e-4d6f-9907-60a21c779c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:37.565301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4eb0b-838e-4d6f-9907-60a21c779c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:37.582896 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4eb0b-838e-4d6f-9907-60a21c779c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:37.585745 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4eb0b-838e-4d6f-9907-60a21c779c7c service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:37.604609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 772/1539] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:14:41.587071 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d06681-12cf-462d-a8b2-0026cfefd9cd None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:41.587837 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d06681-12cf-462d-a8b2-0026cfefd9cd None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:41.587837 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d06681-12cf-462d-a8b2-0026cfefd9cd None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:41.589335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 768/1540] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:14:41.595418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a3f7-e6e3-472e-a3fa-472b5f93b401 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:41.596134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a3f7-e6e3-472e-a3fa-472b5f93b401 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:41.596134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a3f7-e6e3-472e-a3fa-472b5f93b401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:41.617503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-d4a6a3f7-e6e3-472e-a3fa-472b5f93b401 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:41.618746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d4a6a3f7-e6e3-472e-a3fa-472b5f93b401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:41.627040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 773/1541] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:14:41.971314 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d06681-12cf-462d-a8b2-0026cfefd9cd None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:41.973411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-21d06681-12cf-462d-a8b2-0026cfefd9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:42.043288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d06681-12cf-462d-a8b2-0026cfefd9cd None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:42.047046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93c3e6-7381-442e-94b2-0d9413dfeab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:42.047759 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93c3e6-7381-442e-94b2-0d9413dfeab9 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:42.048090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93c3e6-7381-442e-94b2-0d9413dfeab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:42.048428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93c3e6-7381-442e-94b2-0d9413dfeab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:42.060089 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93c3e6-7381-442e-94b2-0d9413dfeab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:42.061760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d93c3e6-7381-442e-94b2-0d9413dfeab9 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:42.073742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 769/1542] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:14:44.492444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22239650-045a-409e-8a8b-32c460ae6939 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:44.493028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22239650-045a-409e-8a8b-32c460ae6939 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:44.493028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22239650-045a-409e-8a8b-32c460ae6939 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:44.494396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 774/1543] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:14:44.500610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65884799-6ec4-4551-abcf-653f7449034d None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:44.501194 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65884799-6ec4-4551-abcf-653f7449034d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:44.501194 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65884799-6ec4-4551-abcf-653f7449034d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:44.522610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-65884799-6ec4-4551-abcf-653f7449034d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:44.523930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-65884799-6ec4-4551-abcf-653f7449034d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:44.532149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 770/1544] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:14:45.086439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22239650-045a-409e-8a8b-32c460ae6939 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:45.088594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-22239650-045a-409e-8a8b-32c460ae6939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:45.098939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=37 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:45.107371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22239650-045a-409e-8a8b-32c460ae6939 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:45.111835 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16573e44-ce90-4f7d-b35f-7228e69a3977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:45.112679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16573e44-ce90-4f7d-b35f-7228e69a3977 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:45.113033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16573e44-ce90-4f7d-b35f-7228e69a3977 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:45.113326 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16573e44-ce90-4f7d-b35f-7228e69a3977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:45.127600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16573e44-ce90-4f7d-b35f-7228e69a3977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:45.129748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16573e44-ce90-4f7d-b35f-7228e69a3977 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:45.142598 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 775/1545] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:14:54.516736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=60 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:55.136555 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:17.904207 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc978c20-b101-4096-8001-25bd7e8df604 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:17.905422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc978c20-b101-4096-8001-25bd7e8df604 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:17.905422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc978c20-b101-4096-8001-25bd7e8df604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:17.934211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-dc978c20-b101-4096-8001-25bd7e8df604 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:17.935579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-dc978c20-b101-4096-8001-25bd7e8df604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:17.951030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 771/1546] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:17.961393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f34b6-3f35-48f5-93cf-2155f14f4737 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:17.962051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f34b6-3f35-48f5-93cf-2155f14f4737 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:17.962051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f34b6-3f35-48f5-93cf-2155f14f4737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:17.993184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-c94f34b6-3f35-48f5-93cf-2155f14f4737 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:17.994763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c94f34b6-3f35-48f5-93cf-2155f14f4737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.003579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b4b55-8dda-4a27-870d-6b2c49ce5d70 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.004048 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b4b55-8dda-4a27-870d-6b2c49ce5d70 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.004304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b4b55-8dda-4a27-870d-6b2c49ce5d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.015062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 776/1547] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.020561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f34b6-3f35-48f5-93cf-2155f14f4737 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.036411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f34b6-3f35-48f5-93cf-2155f14f4737 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.036661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-139b4b55-8dda-4a27-870d-6b2c49ce5d70 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.037993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-139b4b55-8dda-4a27-870d-6b2c49ce5d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.052895 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402fd2d-54ee-4025-9f08-c6446a0b651e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:18.053553 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402fd2d-54ee-4025-9f08-c6446a0b651e admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.053836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402fd2d-54ee-4025-9f08-c6446a0b651e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.055079 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402fd2d-54ee-4025-9f08-c6446a0b651e admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.055079 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402fd2d-54ee-4025-9f08-c6446a0b651e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:16:18.055079 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402fd2d-54ee-4025-9f08-c6446a0b651e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:18.056133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: WARNING py.warnings [None req-c402fd2d-54ee-4025-9f08-c6446a0b651e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:16:18.056133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: warnings.warn(msg) Sep 07 13:16:18.056133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: Sep 07 13:16:18.057039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402fd2d-54ee-4025-9f08-c6446a0b651e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:18.061906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 772/1548] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.078428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6be5b9-2da5-4fc0-9fba-c9e4bd9d2f41 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.078764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6be5b9-2da5-4fc0-9fba-c9e4bd9d2f41 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.078991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6be5b9-2da5-4fc0-9fba-c9e4bd9d2f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.081495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 777/1549] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:16:18.097365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402fd2d-54ee-4025-9f08-c6446a0b651e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.122339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-3c6be5b9-2da5-4fc0-9fba-c9e4bd9d2f41 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.122563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402fd2d-54ee-4025-9f08-c6446a0b651e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.128040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3c6be5b9-2da5-4fc0-9fba-c9e4bd9d2f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.135341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55bb84-9656-4f1e-9320-808522586d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:18.135991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55bb84-9656-4f1e-9320-808522586d71 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.136276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55bb84-9656-4f1e-9320-808522586d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.136550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55bb84-9656-4f1e-9320-808522586d71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.137215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55bb84-9656-4f1e-9320-808522586d71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:18.139882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55bb84-9656-4f1e-9320-808522586d71 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:18.142369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 773/1550] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.148313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6be5b9-2da5-4fc0-9fba-c9e4bd9d2f41 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.167030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6be5b9-2da5-4fc0-9fba-c9e4bd9d2f41 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.185246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9500e1-c767-4f5b-8374-1ffabe343cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:18.186074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9500e1-c767-4f5b-8374-1ffabe343cee admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.186364 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9500e1-c767-4f5b-8374-1ffabe343cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.186634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9500e1-c767-4f5b-8374-1ffabe343cee admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.187340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9500e1-c767-4f5b-8374-1ffabe343cee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:16:18.187645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9500e1-c767-4f5b-8374-1ffabe343cee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:18.189252 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9500e1-c767-4f5b-8374-1ffabe343cee admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:18.193117 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 778/1551] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/projects => generated 389 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:18.201173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55bb84-9656-4f1e-9320-808522586d71 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.214952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 774/1552] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:16:18.223322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d1039-1fc7-4327-a125-fdab5437d699 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.223654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d1039-1fc7-4327-a125-fdab5437d699 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.223873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d1039-1fc7-4327-a125-fdab5437d699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.263632 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-578d1039-1fc7-4327-a125-fdab5437d699 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.265463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-578d1039-1fc7-4327-a125-fdab5437d699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.288905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55bb84-9656-4f1e-9320-808522586d71 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.301478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc2392-b28f-43ef-adc2-40a6c3e92ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:18.302123 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc2392-b28f-43ef-adc2-40a6c3e92ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.302448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc2392-b28f-43ef-adc2-40a6c3e92ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.302662 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc2392-b28f-43ef-adc2-40a6c3e92ad6 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.303410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc2392-b28f-43ef-adc2-40a6c3e92ad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:18.304473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: WARNING py.warnings [None req-8afc2392-b28f-43ef-adc2-40a6c3e92ad6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:16:18.304473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: warnings.warn(msg) Sep 07 13:16:18.304473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: Sep 07 13:16:18.305708 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc2392-b28f-43ef-adc2-40a6c3e92ad6 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:18.339009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 779/1553] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/users => generated 295 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:16:18.339947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc2392-b28f-43ef-adc2-40a6c3e92ad6 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.365012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc2392-b28f-43ef-adc2-40a6c3e92ad6 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.374945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc51e6e-b3b5-4569-981e-5d9286edc4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:18.375618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc51e6e-b3b5-4569-981e-5d9286edc4be admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.375899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc51e6e-b3b5-4569-981e-5d9286edc4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.376170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc51e6e-b3b5-4569-981e-5d9286edc4be admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.376818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc51e6e-b3b5-4569-981e-5d9286edc4be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:18.378780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc51e6e-b3b5-4569-981e-5d9286edc4be admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:18.427833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 775/1554] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.430547 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d1039-1fc7-4327-a125-fdab5437d699 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.434130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 780/1555] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:18.437815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307e43-d07c-4992-afe4-d702d062c945 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.438104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307e43-d07c-4992-afe4-d702d062c945 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.440081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307e43-d07c-4992-afe4-d702d062c945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.456289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d1039-1fc7-4327-a125-fdab5437d699 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.482350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-a5307e43-d07c-4992-afe4-d702d062c945 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.484609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a5307e43-d07c-4992-afe4-d702d062c945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.524935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46635d77-84fd-479a-8a21-9e060f8e2c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:18.525583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46635d77-84fd-479a-8a21-9e060f8e2c9a admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.525859 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46635d77-84fd-479a-8a21-9e060f8e2c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.526126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46635d77-84fd-479a-8a21-9e060f8e2c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.526776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46635d77-84fd-479a-8a21-9e060f8e2c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:18.528430 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46635d77-84fd-479a-8a21-9e060f8e2c9a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:18.538831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 776/1556] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:18.540148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440ef6f-da9d-46b9-9359-c55be3adba05 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.540501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440ef6f-da9d-46b9-9359-c55be3adba05 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.540736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440ef6f-da9d-46b9-9359-c55be3adba05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.562175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-7440ef6f-da9d-46b9-9359-c55be3adba05 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.563469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7440ef6f-da9d-46b9-9359-c55be3adba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.578024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 777/1557] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.579425 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58059e-0cfb-4664-a384-8e278453ebeb None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.579721 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58059e-0cfb-4664-a384-8e278453ebeb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.579976 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58059e-0cfb-4664-a384-8e278453ebeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.582490 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 781/1558] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.583919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5307e43-d07c-4992-afe4-d702d062c945 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.598553 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5307e43-d07c-4992-afe4-d702d062c945 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.603737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80139a36-6158-4e31-a49e-b6cb0223353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:18.604332 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80139a36-6158-4e31-a49e-b6cb0223353a admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.604616 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80139a36-6158-4e31-a49e-b6cb0223353a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.604890 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80139a36-6158-4e31-a49e-b6cb0223353a admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.605539 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80139a36-6158-4e31-a49e-b6cb0223353a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:18.607535 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80139a36-6158-4e31-a49e-b6cb0223353a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:18.616255 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-aa58059e-0cfb-4664-a384-8e278453ebeb None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.617517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-aa58059e-0cfb-4664-a384-8e278453ebeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.628991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 778/1559] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.630373 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b8f77-f46e-484e-b828-2f99e1c373ed None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.630664 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b8f77-f46e-484e-b828-2f99e1c373ed None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.630948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b8f77-f46e-484e-b828-2f99e1c373ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.640304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 782/1560] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/users => generated 288 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:18.643282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdee851-d229-4e03-aca6-09ba5689d575 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.643895 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdee851-d229-4e03-aca6-09ba5689d575 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.644466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdee851-d229-4e03-aca6-09ba5689d575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.653067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-3a9b8f77-f46e-484e-b828-2f99e1c373ed None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.654290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3a9b8f77-f46e-484e-b828-2f99e1c373ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.662694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 779/1561] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.663945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835ac89-19ff-46af-89ea-a1aa27bd42d9 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.664234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835ac89-19ff-46af-89ea-a1aa27bd42d9 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.664609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835ac89-19ff-46af-89ea-a1aa27bd42d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.668597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-6bdee851-d229-4e03-aca6-09ba5689d575 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.669973 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6bdee851-d229-4e03-aca6-09ba5689d575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.678843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 783/1562] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.679566 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdee851-d229-4e03-aca6-09ba5689d575 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.684554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:18.692037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdee851-d229-4e03-aca6-09ba5689d575 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.696645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b835ac89-19ff-46af-89ea-a1aa27bd42d9 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.698397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a7c61-cae7-486d-a93a-1a68c73f08eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:18.699035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b835ac89-19ff-46af-89ea-a1aa27bd42d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.699246 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a7c61-cae7-486d-a93a-1a68c73f08eb admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.699434 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a7c61-cae7-486d-a93a-1a68c73f08eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.699709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a7c61-cae7-486d-a93a-1a68c73f08eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.700393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a7c61-cae7-486d-a93a-1a68c73f08eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:18.702617 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a7c61-cae7-486d-a93a-1a68c73f08eb admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:18.710354 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 780/1563] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.711719 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbfc0e-1e2f-479e-900b-af5d92b8ae1b None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.712003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbfc0e-1e2f-479e-900b-af5d92b8ae1b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.712246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbfc0e-1e2f-479e-900b-af5d92b8ae1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.718117 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 784/1564] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:18.718117 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcde29-2ec9-4ca6-96a9-23629616cb54 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.718117 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcde29-2ec9-4ca6-96a9-23629616cb54 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.718117 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcde29-2ec9-4ca6-96a9-23629616cb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.734012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b3dbfc0e-1e2f-479e-900b-af5d92b8ae1b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.735293 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b3dbfc0e-1e2f-479e-900b-af5d92b8ae1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.742403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-52bcde29-2ec9-4ca6-96a9-23629616cb54 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.743691 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-52bcde29-2ec9-4ca6-96a9-23629616cb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.748867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 781/1565] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.748867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbfc0e-1e2f-479e-900b-af5d92b8ae1b None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.755708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbfc0e-1e2f-479e-900b-af5d92b8ae1b None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.760836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe85ed2-5ccf-4fe8-adff-e44226a805b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:18.761472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe85ed2-5ccf-4fe8-adff-e44226a805b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.761678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 785/1566] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.761789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe85ed2-5ccf-4fe8-adff-e44226a805b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.762674 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcde29-2ec9-4ca6-96a9-23629616cb54 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.766175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe85ed2-5ccf-4fe8-adff-e44226a805b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.766878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe85ed2-5ccf-4fe8-adff-e44226a805b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:16:18.767212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe85ed2-5ccf-4fe8-adff-e44226a805b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:18.768856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe85ed2-5ccf-4fe8-adff-e44226a805b0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:18.773546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcde29-2ec9-4ca6-96a9-23629616cb54 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.778191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3314756-4344-4042-8011-2bae895b79ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:18.778808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3314756-4344-4042-8011-2bae895b79ab admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.779080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3314756-4344-4042-8011-2bae895b79ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.779409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3314756-4344-4042-8011-2bae895b79ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.780836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3314756-4344-4042-8011-2bae895b79ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:18.782486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3314756-4344-4042-8011-2bae895b79ab admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:18.790234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 782/1567] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:16:18.792469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccb9a1-1bed-4b39-af9b-73c90ddcdc77 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.792964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccb9a1-1bed-4b39-af9b-73c90ddcdc77 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.793251 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccb9a1-1bed-4b39-af9b-73c90ddcdc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.800956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 786/1568] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:18.802334 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837c9fd-a671-4ec8-91e7-c0d20f0a0488 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.802624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837c9fd-a671-4ec8-91e7-c0d20f0a0488 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.802900 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837c9fd-a671-4ec8-91e7-c0d20f0a0488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.817698 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-66ccb9a1-1bed-4b39-af9b-73c90ddcdc77 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.819024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-66ccb9a1-1bed-4b39-af9b-73c90ddcdc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.824602 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-2837c9fd-a671-4ec8-91e7-c0d20f0a0488 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.825845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2837c9fd-a671-4ec8-91e7-c0d20f0a0488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.827611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 783/1569] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.828899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbeb1b-4eca-4cc5-9bc2-1ad903dc6623 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.829186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbeb1b-4eca-4cc5-9bc2-1ad903dc6623 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.829464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbeb1b-4eca-4cc5-9bc2-1ad903dc6623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.834520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 787/1570] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.835440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837c9fd-a671-4ec8-91e7-c0d20f0a0488 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.847051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837c9fd-a671-4ec8-91e7-c0d20f0a0488 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.851972 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e76c7-bb06-4578-8310-e793cec7bb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:18.852394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-1cbbeb1b-4eca-4cc5-9bc2-1ad903dc6623 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:18.852686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e76c7-bb06-4578-8310-e793cec7bb8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.852971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e76c7-bb06-4578-8310-e793cec7bb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.853249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e76c7-bb06-4578-8310-e793cec7bb8a admin admin] PATH_INFO: `/v3/projects/3f1963d0789945fcbd182b7f2ce64d03/users/181c888fbd454f00b049522fdfb4197e/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.853632 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1cbbeb1b-4eca-4cc5-9bc2-1ad903dc6623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:18.863063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 784/1571] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:18.866202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbeb1b-4eca-4cc5-9bc2-1ad903dc6623 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.875996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:18.877768 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e76c7-bb06-4578-8310-e793cec7bb8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1963d0789945fcbd182b7f2ce64d03, user_id=181c888fbd454f00b049522fdfb4197e, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:18.880531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbeb1b-4eca-4cc5-9bc2-1ad903dc6623 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.880935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e76c7-bb06-4578-8310-e793cec7bb8a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:18.886482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f043338-2b88-4603-9204-68451f121c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:18.886977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043338-2b88-4603-9204-68451f121c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.887289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043338-2b88-4603-9204-68451f121c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.887681 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043338-2b88-4603-9204-68451f121c06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.888241 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f043338-2b88-4603-9204-68451f121c06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:16:18.888549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f043338-2b88-4603-9204-68451f121c06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:18.890915 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f043338-2b88-4603-9204-68451f121c06 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:18.897030 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 788/1572] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:18 2022] PUT /identity/v3/projects/3f1963d0789945fcbd182b7f2ce64d03/users/181c888fbd454f00b049522fdfb4197e/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:18.897995 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e76c7-bb06-4578-8310-e793cec7bb8a admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.905609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 785/1573] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:16:18.906447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f043338-2b88-4603-9204-68451f121c06 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:18.918502 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f043338-2b88-4603-9204-68451f121c06 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.990093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e76c7-bb06-4578-8310-e793cec7bb8a admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:18.996834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642ccbf-0fc2-4738-bff5-f695d341fe5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:18.997737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642ccbf-0fc2-4738-bff5-f695d341fe5c admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:18.998152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642ccbf-0fc2-4738-bff5-f695d341fe5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:18.998588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642ccbf-0fc2-4738-bff5-f695d341fe5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:18.999615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642ccbf-0fc2-4738-bff5-f695d341fe5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:16:19.000066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642ccbf-0fc2-4738-bff5-f695d341fe5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.002550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642ccbf-0fc2-4738-bff5-f695d341fe5c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.002676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45b0c4-eb93-4a4b-b34d-04bb70d51ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.003679 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45b0c4-eb93-4a4b-b34d-04bb70d51ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.004185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45b0c4-eb93-4a4b-b34d-04bb70d51ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.004686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45b0c4-eb93-4a4b-b34d-04bb70d51ea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.005836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45b0c4-eb93-4a4b-b34d-04bb70d51ea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.006880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: WARNING py.warnings [None req-9f45b0c4-eb93-4a4b-b34d-04bb70d51ea8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:16:19.006880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: warnings.warn(msg) Sep 07 13:16:19.006880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: Sep 07 13:16:19.008149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45b0c4-eb93-4a4b-b34d-04bb70d51ea8 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.021620 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 789/1574] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:16:19.022781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642ccbf-0fc2-4738-bff5-f695d341fe5c admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.040040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642ccbf-0fc2-4738-bff5-f695d341fe5c admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.047527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 786/1575] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:18 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:19.047527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45b0c4-eb93-4a4b-b34d-04bb70d51ea8 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.048793 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5e0be-4f62-4514-b6ae-f12ee30eadc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.049437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5e0be-4f62-4514-b6ae-f12ee30eadc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.049717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5e0be-4f62-4514-b6ae-f12ee30eadc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.049989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5e0be-4f62-4514-b6ae-f12ee30eadc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.050670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5e0be-4f62-4514-b6ae-f12ee30eadc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.052385 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5e0be-4f62-4514-b6ae-f12ee30eadc8 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.067147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 790/1576] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:19.071287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5e0be-4f62-4514-b6ae-f12ee30eadc8 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.082084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5e0be-4f62-4514-b6ae-f12ee30eadc8 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.137030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45b0c4-eb93-4a4b-b34d-04bb70d51ea8 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.144076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585facf-85af-4775-a351-bdb16c881b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.144729 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585facf-85af-4775-a351-bdb16c881b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.145008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585facf-85af-4775-a351-bdb16c881b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.145281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585facf-85af-4775-a351-bdb16c881b8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.146108 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585facf-85af-4775-a351-bdb16c881b8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:16:19.146549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585facf-85af-4775-a351-bdb16c881b8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.149380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585facf-85af-4775-a351-bdb16c881b8c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.171046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 787/1577] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:16:19.171459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e0537-6210-4f96-9330-96e05498c062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.172383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e0537-6210-4f96-9330-96e05498c062 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.172482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585facf-85af-4775-a351-bdb16c881b8c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.172812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e0537-6210-4f96-9330-96e05498c062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.173227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e0537-6210-4f96-9330-96e05498c062 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.174004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e0537-6210-4f96-9330-96e05498c062 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:16:19.174305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e0537-6210-4f96-9330-96e05498c062 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.175977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e0537-6210-4f96-9330-96e05498c062 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.190472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585facf-85af-4775-a351-bdb16c881b8c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.192018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 791/1578] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:16:19.193286 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95579c31-0026-4670-80e3-96b555befd73 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.193571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95579c31-0026-4670-80e3-96b555befd73 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.193841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95579c31-0026-4670-80e3-96b555befd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.196846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353f0a0-34ea-4c83-a809-51170deaac1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.197470 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353f0a0-34ea-4c83-a809-51170deaac1e admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.197817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353f0a0-34ea-4c83-a809-51170deaac1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.198092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353f0a0-34ea-4c83-a809-51170deaac1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.198749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353f0a0-34ea-4c83-a809-51170deaac1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:16:19.199123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353f0a0-34ea-4c83-a809-51170deaac1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.200970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353f0a0-34ea-4c83-a809-51170deaac1e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.214483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 788/1579] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:16:19.215508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353f0a0-34ea-4c83-a809-51170deaac1e admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.216663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-95579c31-0026-4670-80e3-96b555befd73 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:19.218687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-95579c31-0026-4670-80e3-96b555befd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:19.227284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353f0a0-34ea-4c83-a809-51170deaac1e admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.227470 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 792/1580] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:19.232213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95579c31-0026-4670-80e3-96b555befd73 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.237321 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd690a1-bb2b-4bf9-9a44-d957a0057837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.238026 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd690a1-bb2b-4bf9-9a44-d957a0057837 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.238377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd690a1-bb2b-4bf9-9a44-d957a0057837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.238801 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd690a1-bb2b-4bf9-9a44-d957a0057837 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.239548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd690a1-bb2b-4bf9-9a44-d957a0057837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.243428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd690a1-bb2b-4bf9-9a44-d957a0057837 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.247739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95579c31-0026-4670-80e3-96b555befd73 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.258124 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cbd21-266b-4022-90d3-036e0f7885e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.258887 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cbd21-266b-4022-90d3-036e0f7885e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.259304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cbd21-266b-4022-90d3-036e0f7885e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.259563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cbd21-266b-4022-90d3-036e0f7885e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.260426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cbd21-266b-4022-90d3-036e0f7885e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.262848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cbd21-266b-4022-90d3-036e0f7885e1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.277454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:19.285077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 789/1581] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:19 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:19.285077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd690a1-bb2b-4bf9-9a44-d957a0057837 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.301634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 793/1582] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:19 2022] POST /identity/v3/projects => generated 405 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:19.302729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cbd21-266b-4022-90d3-036e0f7885e1 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.327750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:19.355190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd690a1-bb2b-4bf9-9a44-d957a0057837 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.370823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cbd21-266b-4022-90d3-036e0f7885e1 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.376045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a27bf6-de30-43bf-b14a-2b2b005257d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.376758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a27bf6-de30-43bf-b14a-2b2b005257d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.377013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a27bf6-de30-43bf-b14a-2b2b005257d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.377289 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a27bf6-de30-43bf-b14a-2b2b005257d9 admin admin] PATH_INFO: `/v3/projects/ccb52045ffa14e94998e764c1dbc605e/users/b55eed583fac43fb9bfe1366a42bd2fc/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.392809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a27bf6-de30-43bf-b14a-2b2b005257d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb52045ffa14e94998e764c1dbc605e, user_id=b55eed583fac43fb9bfe1366a42bd2fc, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.395774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a27bf6-de30-43bf-b14a-2b2b005257d9 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.414224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 794/1583] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:19 2022] PUT /identity/v3/projects/ccb52045ffa14e94998e764c1dbc605e/users/b55eed583fac43fb9bfe1366a42bd2fc/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:19.421277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a27bf6-de30-43bf-b14a-2b2b005257d9 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.422230 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75959b9d-eb6c-48c3-8f38-4efd7f93a498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.422853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75959b9d-eb6c-48c3-8f38-4efd7f93a498 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.423191 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75959b9d-eb6c-48c3-8f38-4efd7f93a498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.423541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75959b9d-eb6c-48c3-8f38-4efd7f93a498 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.424188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75959b9d-eb6c-48c3-8f38-4efd7f93a498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.426448 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75959b9d-eb6c-48c3-8f38-4efd7f93a498 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.450519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 790/1584] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:19 2022] POST /identity/v3/users => generated 291 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:19.451387 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75959b9d-eb6c-48c3-8f38-4efd7f93a498 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.461799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75959b9d-eb6c-48c3-8f38-4efd7f93a498 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.490786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a27bf6-de30-43bf-b14a-2b2b005257d9 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.497646 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1dd0e-250a-4fb4-b067-b7e180294eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.498219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1dd0e-250a-4fb4-b067-b7e180294eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.498492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1dd0e-250a-4fb4-b067-b7e180294eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.498766 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1dd0e-250a-4fb4-b067-b7e180294eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.499448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1dd0e-250a-4fb4-b067-b7e180294eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.501388 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1dd0e-250a-4fb4-b067-b7e180294eb7 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.525533 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93ef71-c3eb-41f2-8830-4f48f16668b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.526156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93ef71-c3eb-41f2-8830-4f48f16668b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.526436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93ef71-c3eb-41f2-8830-4f48f16668b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.526708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93ef71-c3eb-41f2-8830-4f48f16668b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.527428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93ef71-c3eb-41f2-8830-4f48f16668b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.530162 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 795/1585] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:19 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:19.530929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1dd0e-250a-4fb4-b067-b7e180294eb7 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.533435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93ef71-c3eb-41f2-8830-4f48f16668b8 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.562672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 791/1586] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:19 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:19.563655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93ef71-c3eb-41f2-8830-4f48f16668b8 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.625928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1dd0e-250a-4fb4-b067-b7e180294eb7 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.630685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:19.633752 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93ef71-c3eb-41f2-8830-4f48f16668b8 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.637750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da3786-f2c2-41b4-bbc8-2161a0baa1e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.638364 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da3786-f2c2-41b4-bbc8-2161a0baa1e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.638647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da3786-f2c2-41b4-bbc8-2161a0baa1e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.639479 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da3786-f2c2-41b4-bbc8-2161a0baa1e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.650954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:19.699449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b01cab-e1c2-4599-82b7-995db827353e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.700074 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b01cab-e1c2-4599-82b7-995db827353e admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.700363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b01cab-e1c2-4599-82b7-995db827353e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.700636 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b01cab-e1c2-4599-82b7-995db827353e admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.701276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b01cab-e1c2-4599-82b7-995db827353e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.704018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b01cab-e1c2-4599-82b7-995db827353e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.720332 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da3786-f2c2-41b4-bbc8-2161a0baa1e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.722564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da3786-f2c2-41b4-bbc8-2161a0baa1e7 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.736979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 796/1587] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:19 2022] POST /identity/v3/projects => generated 393 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:19.736979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b01cab-e1c2-4599-82b7-995db827353e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.737274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 792/1588] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:19.737274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da3786-f2c2-41b4-bbc8-2161a0baa1e7 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.746754 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da3786-f2c2-41b4-bbc8-2161a0baa1e7 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.820615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b01cab-e1c2-4599-82b7-995db827353e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.826944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcd4b6-7273-4b1a-b98d-b52f9dcf78c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.827208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44550a-846d-432b-8653-ba5078ae07dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.828036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd4b6-7273-4b1a-b98d-b52f9dcf78c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.828190 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44550a-846d-432b-8653-ba5078ae07dd admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.828456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44550a-846d-432b-8653-ba5078ae07dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.828721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44550a-846d-432b-8653-ba5078ae07dd admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.828936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd4b6-7273-4b1a-b98d-b52f9dcf78c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.829384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd4b6-7273-4b1a-b98d-b52f9dcf78c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.829642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44550a-846d-432b-8653-ba5078ae07dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.830393 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcd4b6-7273-4b1a-b98d-b52f9dcf78c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.832075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44550a-846d-432b-8653-ba5078ae07dd admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.832962 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcd4b6-7273-4b1a-b98d-b52f9dcf78c4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.864255 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 793/1589] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:19 2022] POST /identity/v3/users => generated 303 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:19.867336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 797/1590] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:19 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:19.867336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44550a-846d-432b-8653-ba5078ae07dd admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.867559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198102b-6985-4d74-a5a2-bb77c0c28612 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.867559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198102b-6985-4d74-a5a2-bb77c0c28612 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.867559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198102b-6985-4d74-a5a2-bb77c0c28612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.880083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44550a-846d-432b-8653-ba5078ae07dd admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.889860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b055e0-4461-4a11-bb7a-7ba1d032f287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.890703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b055e0-4461-4a11-bb7a-7ba1d032f287 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.891571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b055e0-4461-4a11-bb7a-7ba1d032f287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.891937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b055e0-4461-4a11-bb7a-7ba1d032f287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.892985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-4198102b-6985-4d74-a5a2-bb77c0c28612 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:19.893116 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b055e0-4461-4a11-bb7a-7ba1d032f287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.894506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4198102b-6985-4d74-a5a2-bb77c0c28612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:19.895545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b055e0-4461-4a11-bb7a-7ba1d032f287 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.907269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 794/1591] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:19.907269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198102b-6985-4d74-a5a2-bb77c0c28612 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.918286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198102b-6985-4d74-a5a2-bb77c0c28612 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.924571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 798/1592] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:19.924755 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ab2b5-76f1-43fd-a912-7ca7d92c7bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.925333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ab2b5-76f1-43fd-a912-7ca7d92c7bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.926013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ab2b5-76f1-43fd-a912-7ca7d92c7bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.926352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ab2b5-76f1-43fd-a912-7ca7d92c7bb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.927064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ab2b5-76f1-43fd-a912-7ca7d92c7bb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.928544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b055e0-4461-4a11-bb7a-7ba1d032f287 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.929458 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ab2b5-76f1-43fd-a912-7ca7d92c7bb1 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.939872 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b055e0-4461-4a11-bb7a-7ba1d032f287 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.946651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5c7e8-f2cf-463e-bff1-352d3eb4e415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.947457 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5c7e8-f2cf-463e-bff1-352d3eb4e415 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.947731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5c7e8-f2cf-463e-bff1-352d3eb4e415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.948001 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5c7e8-f2cf-463e-bff1-352d3eb4e415 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.948637 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5c7e8-f2cf-463e-bff1-352d3eb4e415 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.950587 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5c7e8-f2cf-463e-bff1-352d3eb4e415 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.957851 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 795/1593] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:19 2022] POST /identity/v3/users => generated 289 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:19.957851 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ab2b5-76f1-43fd-a912-7ca7d92c7bb1 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.970801 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ab2b5-76f1-43fd-a912-7ca7d92c7bb1 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.975613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 799/1594] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:19 2022] POST /identity/v3/users => generated 288 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:19.975912 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6727df-50d4-44b8-b2c0-1dc429c7119a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.976503 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6727df-50d4-44b8-b2c0-1dc429c7119a admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.976871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6727df-50d4-44b8-b2c0-1dc429c7119a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.977150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6727df-50d4-44b8-b2c0-1dc429c7119a admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.977271 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5c7e8-f2cf-463e-bff1-352d3eb4e415 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:19.977821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6727df-50d4-44b8-b2c0-1dc429c7119a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:19.979790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6727df-50d4-44b8-b2c0-1dc429c7119a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:19.991699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5c7e8-f2cf-463e-bff1-352d3eb4e415 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:19.997335 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebee966-88f4-45f6-a3ad-b3b7e879c11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:19.997896 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebee966-88f4-45f6-a3ad-b3b7e879c11e admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:19.998702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebee966-88f4-45f6-a3ad-b3b7e879c11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:19.999016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebee966-88f4-45f6-a3ad-b3b7e879c11e admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:19.999717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebee966-88f4-45f6-a3ad-b3b7e879c11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.001438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebee966-88f4-45f6-a3ad-b3b7e879c11e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.004031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 796/1595] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:19 2022] POST /identity/v3/users => generated 297 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:20.004758 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6727df-50d4-44b8-b2c0-1dc429c7119a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.016176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6727df-50d4-44b8-b2c0-1dc429c7119a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.017017 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 800/1596] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:20.017895 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebee966-88f4-45f6-a3ad-b3b7e879c11e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.020170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-4ebee966-88f4-45f6-a3ad-b3b7e879c11e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:20.024420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c1712-e414-493e-ae9c-141b04418d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.025229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c1712-e414-493e-ae9c-141b04418d33 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.025728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c1712-e414-493e-ae9c-141b04418d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.026008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c1712-e414-493e-ae9c-141b04418d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.026671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c1712-e414-493e-ae9c-141b04418d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.028360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c1712-e414-493e-ae9c-141b04418d33 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.041059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 797/1597] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:20.042165 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c1712-e414-493e-ae9c-141b04418d33 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.053808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c1712-e414-493e-ae9c-141b04418d33 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.059570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db411489-199b-4818-ada1-452d65ab86a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.060195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db411489-199b-4818-ada1-452d65ab86a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.060480 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db411489-199b-4818-ada1-452d65ab86a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.061298 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db411489-199b-4818-ada1-452d65ab86a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.062119 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db411489-199b-4818-ada1-452d65ab86a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.063812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db411489-199b-4818-ada1-452d65ab86a0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.079637 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:20.087489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 798/1598] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:20.088822 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db411489-199b-4818-ada1-452d65ab86a0 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.093469 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebee966-88f4-45f6-a3ad-b3b7e879c11e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.100039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650171f-019b-421e-b6d8-b7b5cf3a585f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.100868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650171f-019b-421e-b6d8-b7b5cf3a585f service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.101532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650171f-019b-421e-b6d8-b7b5cf3a585f service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.102291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650171f-019b-421e-b6d8-b7b5cf3a585f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.104998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db411489-199b-4818-ada1-452d65ab86a0 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.110746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac53ef-0487-4f02-afcd-50dafde6c320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.111168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac53ef-0487-4f02-afcd-50dafde6c320 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.111714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac53ef-0487-4f02-afcd-50dafde6c320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.111980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac53ef-0487-4f02-afcd-50dafde6c320 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.112780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac53ef-0487-4f02-afcd-50dafde6c320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.115831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac53ef-0487-4f02-afcd-50dafde6c320 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.117181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650171f-019b-421e-b6d8-b7b5cf3a585f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.120974 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650171f-019b-421e-b6d8-b7b5cf3a585f service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.129562 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 799/1599] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:20.130424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac53ef-0487-4f02-afcd-50dafde6c320 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.143183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 801/1600] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:20.144655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac53ef-0487-4f02-afcd-50dafde6c320 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.154792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650171f-019b-421e-b6d8-b7b5cf3a585f service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.155364 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673eb9dc-f101-4098-824b-f4cc725e90fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.155972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb9dc-f101-4098-824b-f4cc725e90fe admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.156427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb9dc-f101-4098-824b-f4cc725e90fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.156708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb9dc-f101-4098-824b-f4cc725e90fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.157364 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673eb9dc-f101-4098-824b-f4cc725e90fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.159013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673eb9dc-f101-4098-824b-f4cc725e90fe admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.173874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 800/1601] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:20.174067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673eb9dc-f101-4098-824b-f4cc725e90fe admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.181235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650171f-019b-421e-b6d8-b7b5cf3a585f service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.187449 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673eb9dc-f101-4098-824b-f4cc725e90fe admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.189327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e5df3-f726-410b-8d09-7f9624e39f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.190307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e5df3-f726-410b-8d09-7f9624e39f6e admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.190770 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e5df3-f726-410b-8d09-7f9624e39f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.191310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e5df3-f726-410b-8d09-7f9624e39f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.192396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e5df3-f726-410b-8d09-7f9624e39f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.193332 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a7a6e-e7f6-41dc-9d6b-b5f7ac81c14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.194074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a7a6e-e7f6-41dc-9d6b-b5f7ac81c14e admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.194447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a7a6e-e7f6-41dc-9d6b-b5f7ac81c14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.194735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e5df3-f726-410b-8d09-7f9624e39f6e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.195078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a7a6e-e7f6-41dc-9d6b-b5f7ac81c14e admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.195920 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a7a6e-e7f6-41dc-9d6b-b5f7ac81c14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.197918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a7a6e-e7f6-41dc-9d6b-b5f7ac81c14e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.207813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:20.215092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 802/1602] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:20.215092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e5df3-f726-410b-8d09-7f9624e39f6e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.217266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 801/1603] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:20.219525 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a7a6e-e7f6-41dc-9d6b-b5f7ac81c14e admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.228918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e5df3-f726-410b-8d09-7f9624e39f6e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.235158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad0a9c-b805-471c-b982-be162dabc283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.235770 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad0a9c-b805-471c-b982-be162dabc283 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.236061 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad0a9c-b805-471c-b982-be162dabc283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.236603 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad0a9c-b805-471c-b982-be162dabc283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.237260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad0a9c-b805-471c-b982-be162dabc283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.239301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad0a9c-b805-471c-b982-be162dabc283 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.241964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a7a6e-e7f6-41dc-9d6b-b5f7ac81c14e admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.249997 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ca780-426d-4a02-bdb9-9129629641e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.250901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca780-426d-4a02-bdb9-9129629641e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.251370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca780-426d-4a02-bdb9-9129629641e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.251751 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca780-426d-4a02-bdb9-9129629641e3 admin admin] PATH_INFO: `/v3/projects/52f78f78a6054c53bc715e02f4a02f30/users/5f8f8d47ecd646dca1362021e70657b6/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.255340 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 803/1604] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:20.255340 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad0a9c-b805-471c-b982-be162dabc283 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.269307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad0a9c-b805-471c-b982-be162dabc283 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.276531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ca780-426d-4a02-bdb9-9129629641e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f78f78a6054c53bc715e02f4a02f30, user_id=5f8f8d47ecd646dca1362021e70657b6, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.276801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94713b63-97e9-4cbc-bf39-8370c3bee546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.277362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94713b63-97e9-4cbc-bf39-8370c3bee546 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.277711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94713b63-97e9-4cbc-bf39-8370c3bee546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.277999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94713b63-97e9-4cbc-bf39-8370c3bee546 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.278154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: WARNING py.warnings [None req-669ca780-426d-4a02-bdb9-9129629641e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:16:20.278154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: warnings.warn(msg) Sep 07 13:16:20.278154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: Sep 07 13:16:20.278646 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94713b63-97e9-4cbc-bf39-8370c3bee546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.280999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94713b63-97e9-4cbc-bf39-8370c3bee546 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.281526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ca780-426d-4a02-bdb9-9129629641e3 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.297719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 804/1605] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:20.299026 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94713b63-97e9-4cbc-bf39-8370c3bee546 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.300839 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 802/1606] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:20 2022] PUT /identity/v3/projects/52f78f78a6054c53bc715e02f4a02f30/users/5f8f8d47ecd646dca1362021e70657b6/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:20.301800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ca780-426d-4a02-bdb9-9129629641e3 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.309228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94713b63-97e9-4cbc-bf39-8370c3bee546 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.385221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba972df0-f17e-4a39-ae1a-edd485d71c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.385853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba972df0-f17e-4a39-ae1a-edd485d71c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.386121 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba972df0-f17e-4a39-ae1a-edd485d71c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.386390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba972df0-f17e-4a39-ae1a-edd485d71c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.387041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba972df0-f17e-4a39-ae1a-edd485d71c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.388740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba972df0-f17e-4a39-ae1a-edd485d71c12 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.400899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 805/1607] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:20.401912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba972df0-f17e-4a39-ae1a-edd485d71c12 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.407040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ca780-426d-4a02-bdb9-9129629641e3 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.412987 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba972df0-f17e-4a39-ae1a-edd485d71c12 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.414050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dae303-d002-46fb-9e93-421a8bfde966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.415032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dae303-d002-46fb-9e93-421a8bfde966 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.415563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dae303-d002-46fb-9e93-421a8bfde966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.415851 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dae303-d002-46fb-9e93-421a8bfde966 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.416533 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dae303-d002-46fb-9e93-421a8bfde966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.417964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67409895-1232-4f8f-9854-5a4626a39844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.418183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dae303-d002-46fb-9e93-421a8bfde966 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.418784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67409895-1232-4f8f-9854-5a4626a39844 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.419159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67409895-1232-4f8f-9854-5a4626a39844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.419501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67409895-1232-4f8f-9854-5a4626a39844 admin admin] PATH_INFO: `/v3/projects/06e87ad5726745d5a06276f0eb82788a/users/8f6db3f4d7df41c5be5a22a753c5606d/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.435349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 803/1608] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:20.435349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dae303-d002-46fb-9e93-421a8bfde966 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.439773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67409895-1232-4f8f-9854-5a4626a39844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e87ad5726745d5a06276f0eb82788a, user_id=8f6db3f4d7df41c5be5a22a753c5606d, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.443590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67409895-1232-4f8f-9854-5a4626a39844 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.444183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dae303-d002-46fb-9e93-421a8bfde966 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.449379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373e5a1-a18f-4be7-af9b-ecdd64ac8638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.450376 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373e5a1-a18f-4be7-af9b-ecdd64ac8638 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.450988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373e5a1-a18f-4be7-af9b-ecdd64ac8638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.451954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373e5a1-a18f-4be7-af9b-ecdd64ac8638 admin admin] PATH_INFO: `/v3/projects/ddc95f1674564cb7b1d3297a67a85d02/users/c936ea7e2f224d03926a2004476dff5a/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.460307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 806/1609] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:20 2022] PUT /identity/v3/projects/06e87ad5726745d5a06276f0eb82788a/users/8f6db3f4d7df41c5be5a22a753c5606d/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:20.462056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67409895-1232-4f8f-9854-5a4626a39844 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.474268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373e5a1-a18f-4be7-af9b-ecdd64ac8638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddc95f1674564cb7b1d3297a67a85d02, user_id=c936ea7e2f224d03926a2004476dff5a, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.478078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373e5a1-a18f-4be7-af9b-ecdd64ac8638 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.498421 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 804/1610] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:20 2022] PUT /identity/v3/projects/ddc95f1674564cb7b1d3297a67a85d02/users/c936ea7e2f224d03926a2004476dff5a/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:20.499777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732fc9d-c9b8-4a33-a09a-2e3514849ad7 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.500063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732fc9d-c9b8-4a33-a09a-2e3514849ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.500335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732fc9d-c9b8-4a33-a09a-2e3514849ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.543570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b732fc9d-c9b8-4a33-a09a-2e3514849ad7 None None] MFA Rules not processed for user `5f8f8d47ecd646dca1362021e70657b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:20.544980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b732fc9d-c9b8-4a33-a09a-2e3514849ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:20.546248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67409895-1232-4f8f-9854-5a4626a39844 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.585623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:20.620099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832013a-3436-4b4d-b520-761fc7b0cbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.621154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832013a-3436-4b4d-b520-761fc7b0cbe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.621604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832013a-3436-4b4d-b520-761fc7b0cbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.621920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832013a-3436-4b4d-b520-761fc7b0cbe4 admin admin] PATH_INFO: `/v3/projects/d43169ae236d4341abb683010449c5e2/users/cf718a19fbc9458c996678ed7accc344/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.649104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832013a-3436-4b4d-b520-761fc7b0cbe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d43169ae236d4341abb683010449c5e2, user_id=cf718a19fbc9458c996678ed7accc344, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.649104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832013a-3436-4b4d-b520-761fc7b0cbe4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.665615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 807/1611] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:20 2022] PUT /identity/v3/projects/d43169ae236d4341abb683010449c5e2/users/cf718a19fbc9458c996678ed7accc344/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:20.666722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832013a-3436-4b4d-b520-761fc7b0cbe4 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.671813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 805/1612] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:20.672938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732fc9d-c9b8-4a33-a09a-2e3514849ad7 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:20.694509 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732fc9d-c9b8-4a33-a09a-2e3514849ad7 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.744213 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:20.761153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832013a-3436-4b4d-b520-761fc7b0cbe4 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:20.770179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0715e-ac6e-4936-83d9-a5f65f12c8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.771002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0715e-ac6e-4936-83d9-a5f65f12c8ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.771530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0715e-ac6e-4936-83d9-a5f65f12c8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.771814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0715e-ac6e-4936-83d9-a5f65f12c8ff admin admin] PATH_INFO: `/v3/projects/267fd08403b049f0b6464921bc8fa2f3/users/cc2142713d524f4da090c4fc72b25db9/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.778629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23a404-f22c-4dd4-8122-1cd53c646862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:20.779337 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23a404-f22c-4dd4-8122-1cd53c646862 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.779653 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23a404-f22c-4dd4-8122-1cd53c646862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.780078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23a404-f22c-4dd4-8122-1cd53c646862 admin admin] PATH_INFO: `/v3/projects/2d2ca3f0240b478e8463f74032a1b575/users/406bdf61f9284ebc8620b21ee414ec91/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.791675 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0715e-ac6e-4936-83d9-a5f65f12c8ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=267fd08403b049f0b6464921bc8fa2f3, user_id=cc2142713d524f4da090c4fc72b25db9, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.794946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0715e-ac6e-4936-83d9-a5f65f12c8ff admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.809529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23a404-f22c-4dd4-8122-1cd53c646862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d2ca3f0240b478e8463f74032a1b575, user_id=406bdf61f9284ebc8620b21ee414ec91, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:20.814451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23a404-f22c-4dd4-8122-1cd53c646862 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:20.818229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 808/1613] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:20 2022] PUT /identity/v3/projects/267fd08403b049f0b6464921bc8fa2f3/users/cc2142713d524f4da090c4fc72b25db9/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:20.820014 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c22ac-2017-4941-b8d9-aa0d0fc1b914 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.820243 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c22ac-2017-4941-b8d9-aa0d0fc1b914 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.820600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c22ac-2017-4941-b8d9-aa0d0fc1b914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.848211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 806/1614] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:20 2022] PUT /identity/v3/projects/2d2ca3f0240b478e8463f74032a1b575/users/406bdf61f9284ebc8620b21ee414ec91/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:20.850441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e0312-cad1-4c7c-af08-4023a17c3341 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:20.852309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e0312-cad1-4c7c-af08-4023a17c3341 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:20.853013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e0312-cad1-4c7c-af08-4023a17c3341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:20.858140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-c01c22ac-2017-4941-b8d9-aa0d0fc1b914 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:20.860654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c01c22ac-2017-4941-b8d9-aa0d0fc1b914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:20.906985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-537e0312-cad1-4c7c-af08-4023a17c3341 None None] MFA Rules not processed for user `c936ea7e2f224d03926a2004476dff5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:20.909607 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-537e0312-cad1-4c7c-af08-4023a17c3341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:20.995746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 809/1615] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:21.000035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb4e8a-0390-4080-9b7e-9f831722a6a3 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.000035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb4e8a-0390-4080-9b7e-9f831722a6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.000035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb4e8a-0390-4080-9b7e-9f831722a6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.023253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8feb4e8a-0390-4080-9b7e-9f831722a6a3 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:21.024748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8feb4e8a-0390-4080-9b7e-9f831722a6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:21.037370 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 810/1616] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:21.038589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb4e8a-0390-4080-9b7e-9f831722a6a3 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:21.051669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb4e8a-0390-4080-9b7e-9f831722a6a3 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:21.053086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 807/1617] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:21.054539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334d876-6096-4e2a-a44f-a61b48d97c9a None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.054873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334d876-6096-4e2a-a44f-a61b48d97c9a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.058240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334d876-6096-4e2a-a44f-a61b48d97c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.063499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164853e-7363-4f7f-a945-0c5e8790ff12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:21.064264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164853e-7363-4f7f-a945-0c5e8790ff12 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.064641 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164853e-7363-4f7f-a945-0c5e8790ff12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.065033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164853e-7363-4f7f-a945-0c5e8790ff12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.065798 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164853e-7363-4f7f-a945-0c5e8790ff12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:21.068252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164853e-7363-4f7f-a945-0c5e8790ff12 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:21.110837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 811/1618] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:21 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:21.113060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5144752-6f41-4fc0-ad3f-a250105e99b9 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.113256 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5144752-6f41-4fc0-ad3f-a250105e99b9 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.113527 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5144752-6f41-4fc0-ad3f-a250105e99b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.132585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-9334d876-6096-4e2a-a44f-a61b48d97c9a None None] MFA Rules not processed for user `cc2142713d524f4da090c4fc72b25db9`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:21.134613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9334d876-6096-4e2a-a44f-a61b48d97c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:21.135535 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-b5144752-6f41-4fc0-ad3f-a250105e99b9 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:21.136729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b5144752-6f41-4fc0-ad3f-a250105e99b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:21.142529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:21.214510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 812/1619] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:21.214510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5144752-6f41-4fc0-ad3f-a250105e99b9 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:21.273365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 808/1620] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:21.274426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334d876-6096-4e2a-a44f-a61b48d97c9a None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:21.290689 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334d876-6096-4e2a-a44f-a61b48d97c9a None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:21.303384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd79acc-a821-4b10-9899-bd3d2289257d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:21.304005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd79acc-a821-4b10-9899-bd3d2289257d admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.304293 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd79acc-a821-4b10-9899-bd3d2289257d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.304579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd79acc-a821-4b10-9899-bd3d2289257d admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.305387 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd79acc-a821-4b10-9899-bd3d2289257d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:21.307813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd79acc-a821-4b10-9899-bd3d2289257d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:21.331044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5144752-6f41-4fc0-ad3f-a250105e99b9 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:21.344943 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ada1df-79ca-4e94-9400-63d2d1a0f696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:21.347013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ada1df-79ca-4e94-9400-63d2d1a0f696 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.347529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ada1df-79ca-4e94-9400-63d2d1a0f696 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.348722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ada1df-79ca-4e94-9400-63d2d1a0f696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.354172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 809/1621] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:21 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:21.355233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd79acc-a821-4b10-9899-bd3d2289257d admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:21.372910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ada1df-79ca-4e94-9400-63d2d1a0f696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:21.375340 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ada1df-79ca-4e94-9400-63d2d1a0f696 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:21.387879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 813/1622] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:21.388818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ada1df-79ca-4e94-9400-63d2d1a0f696 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:21.393050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:21.402339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ada1df-79ca-4e94-9400-63d2d1a0f696 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:21.469508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd79acc-a821-4b10-9899-bd3d2289257d admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:21.476099 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff25e4e-747f-4ba8-b4f4-ed8c48885c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:21.477655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff25e4e-747f-4ba8-b4f4-ed8c48885c27 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.478285 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff25e4e-747f-4ba8-b4f4-ed8c48885c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.478594 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff25e4e-747f-4ba8-b4f4-ed8c48885c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.488894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1d1db-1757-41be-8305-3c2ec2a75cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:21.490437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1d1db-1757-41be-8305-3c2ec2a75cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.490921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1d1db-1757-41be-8305-3c2ec2a75cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.491393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1d1db-1757-41be-8305-3c2ec2a75cb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.492255 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1d1db-1757-41be-8305-3c2ec2a75cb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:21.495414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1d1db-1757-41be-8305-3c2ec2a75cb3 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:21.535294 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 814/1623] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:21 2022] POST /identity/v3/users => generated 297 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:21.536924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1d1db-1757-41be-8305-3c2ec2a75cb3 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:21.550175 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1d1db-1757-41be-8305-3c2ec2a75cb3 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:21.555950 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09a1f0-8c27-4511-8401-d02454099df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:21.556661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09a1f0-8c27-4511-8401-d02454099df0 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.557435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09a1f0-8c27-4511-8401-d02454099df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.558082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09a1f0-8c27-4511-8401-d02454099df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.566654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff25e4e-747f-4ba8-b4f4-ed8c48885c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:21.569899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff25e4e-747f-4ba8-b4f4-ed8c48885c27 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:21.570842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09a1f0-8c27-4511-8401-d02454099df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:21.573060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09a1f0-8c27-4511-8401-d02454099df0 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:21.584311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 810/1624] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:21.585983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948711e4-653b-47c0-98d7-4fb311ab6f08 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.586485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948711e4-653b-47c0-98d7-4fb311ab6f08 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.586953 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948711e4-653b-47c0-98d7-4fb311ab6f08 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.588649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 811/1625] 10.222.0.61 () {54 vars in 962 bytes} [Wed Sep 7 13:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:16:21.590090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948711e4-653b-47c0-98d7-4fb311ab6f08 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:21.594394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 815/1626] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:21.595443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09a1f0-8c27-4511-8401-d02454099df0 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:21.607191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09a1f0-8c27-4511-8401-d02454099df0 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:21.611805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948711e4-653b-47c0-98d7-4fb311ab6f08 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:21.613474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810394fe-900d-4d54-9d1a-9236b0ea8e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:21.614209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810394fe-900d-4d54-9d1a-9236b0ea8e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.614557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810394fe-900d-4d54-9d1a-9236b0ea8e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.614898 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810394fe-900d-4d54-9d1a-9236b0ea8e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.615608 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810394fe-900d-4d54-9d1a-9236b0ea8e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:21.617413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810394fe-900d-4d54-9d1a-9236b0ea8e90 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:21.617974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86266652-5a7e-4b95-bff6-f6b4c3e4869d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:21.618766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86266652-5a7e-4b95-bff6-f6b4c3e4869d admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.619387 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86266652-5a7e-4b95-bff6-f6b4c3e4869d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.619769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86266652-5a7e-4b95-bff6-f6b4c3e4869d admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.620857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86266652-5a7e-4b95-bff6-f6b4c3e4869d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:21.623901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86266652-5a7e-4b95-bff6-f6b4c3e4869d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:21.634658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 816/1627] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:21.636156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf06e6-18a1-481c-ac71-db8963e4e6ad None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.636495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf06e6-18a1-481c-ac71-db8963e4e6ad None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.636918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf06e6-18a1-481c-ac71-db8963e4e6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.661392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 812/1628] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:21 2022] POST /identity/v3/users => generated 293 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:21.662622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86266652-5a7e-4b95-bff6-f6b4c3e4869d admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:21.679979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86266652-5a7e-4b95-bff6-f6b4c3e4869d admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:21.691401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a3787-2534-430c-9bb4-52fea8638daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:21.692853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a3787-2534-430c-9bb4-52fea8638daa admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.693423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a3787-2534-430c-9bb4-52fea8638daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.693954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a3787-2534-430c-9bb4-52fea8638daa admin admin] PATH_INFO: `/v3/projects/1088a8fb7f2f4544a00955c5ba7d2654/users/6ef33be1b1c44218b33931e8eb99b577/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.718956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a3787-2534-430c-9bb4-52fea8638daa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1088a8fb7f2f4544a00955c5ba7d2654, user_id=6ef33be1b1c44218b33931e8eb99b577, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:21.719190 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:21.722710 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a3787-2534-430c-9bb4-52fea8638daa admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:21.738693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-61cf06e6-18a1-481c-ac71-db8963e4e6ad None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:21.744174 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-61cf06e6-18a1-481c-ac71-db8963e4e6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:21.753858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 813/1629] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:21 2022] PUT /identity/v3/projects/1088a8fb7f2f4544a00955c5ba7d2654/users/6ef33be1b1c44218b33931e8eb99b577/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:21.755357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a3787-2534-430c-9bb4-52fea8638daa admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:21.866193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a3787-2534-430c-9bb4-52fea8638daa admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:21.874212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923765f3-3a61-4f38-8539-ddc219da9833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:21.875272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923765f3-3a61-4f38-8539-ddc219da9833 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.875773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923765f3-3a61-4f38-8539-ddc219da9833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.876047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923765f3-3a61-4f38-8539-ddc219da9833 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.876758 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923765f3-3a61-4f38-8539-ddc219da9833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:21.880389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923765f3-3a61-4f38-8539-ddc219da9833 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:21.905234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 814/1630] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:21.905234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923765f3-3a61-4f38-8539-ddc219da9833 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:21.918426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923765f3-3a61-4f38-8539-ddc219da9833 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:21.920963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 817/1631] 10.222.0.61 () {58 vars in 1059 bytes} [Wed Sep 7 13:16:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:21.922044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf06e6-18a1-481c-ac71-db8963e4e6ad None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:21.925278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543cbe7-cf85-4533-80bf-8da42bdb3a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:21.925916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543cbe7-cf85-4533-80bf-8da42bdb3a1e admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.926372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543cbe7-cf85-4533-80bf-8da42bdb3a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.926716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543cbe7-cf85-4533-80bf-8da42bdb3a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.927669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543cbe7-cf85-4533-80bf-8da42bdb3a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:21.929174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543cbe7-cf85-4533-80bf-8da42bdb3a1e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:21.948221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 815/1632] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:21.949122 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543cbe7-cf85-4533-80bf-8da42bdb3a1e admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:21.952556 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf06e6-18a1-481c-ac71-db8963e4e6ad None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:21.959757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0804315-adf1-4a6f-9084-6fdf565e0c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:21.961137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0804315-adf1-4a6f-9084-6fdf565e0c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.961733 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0804315-adf1-4a6f-9084-6fdf565e0c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.962735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0804315-adf1-4a6f-9084-6fdf565e0c25 admin admin] PATH_INFO: `/v3/projects/43a3b8964d994776817aa8528658714b/users/a7f1edf1fa6544079804e47c2c1939d2/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.969743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543cbe7-cf85-4533-80bf-8da42bdb3a1e admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:21.976993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a1826-b62f-4119-8cee-cf54067ea165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:21.978327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a1826-b62f-4119-8cee-cf54067ea165 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:21.978566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a1826-b62f-4119-8cee-cf54067ea165 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:21.978899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a1826-b62f-4119-8cee-cf54067ea165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:21.992324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:22.017591 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0804315-adf1-4a6f-9084-6fdf565e0c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43a3b8964d994776817aa8528658714b, user_id=a7f1edf1fa6544079804e47c2c1939d2, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:22.020577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0804315-adf1-4a6f-9084-6fdf565e0c25 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:22.050725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 818/1633] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:21 2022] PUT /identity/v3/projects/43a3b8964d994776817aa8528658714b/users/a7f1edf1fa6544079804e47c2c1939d2/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:22.050725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0804315-adf1-4a6f-9084-6fdf565e0c25 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:22.060457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a1826-b62f-4119-8cee-cf54067ea165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:22.063082 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a1826-b62f-4119-8cee-cf54067ea165 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:22.075817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 816/1634] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:22.077200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a1826-b62f-4119-8cee-cf54067ea165 service nova] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:22.089786 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a1826-b62f-4119-8cee-cf54067ea165 service nova] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:22.165135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5fd7c-75e1-45ad-8387-280ef8eddb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:22.165647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5fd7c-75e1-45ad-8387-280ef8eddb1f admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:22.165985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5fd7c-75e1-45ad-8387-280ef8eddb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:22.166267 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5fd7c-75e1-45ad-8387-280ef8eddb1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:22.167314 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5fd7c-75e1-45ad-8387-280ef8eddb1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:22.169040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5fd7c-75e1-45ad-8387-280ef8eddb1f admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:22.186706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 817/1635] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:22.189642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5fd7c-75e1-45ad-8387-280ef8eddb1f admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:22.200652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0804315-adf1-4a6f-9084-6fdf565e0c25 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:22.202286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5fd7c-75e1-45ad-8387-280ef8eddb1f admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:22.210030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff066c-2c46-477a-9167-133e162cee80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:22.210880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff066c-2c46-477a-9167-133e162cee80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:22.211336 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff066c-2c46-477a-9167-133e162cee80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:22.211717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff066c-2c46-477a-9167-133e162cee80 admin admin] PATH_INFO: `/v3/projects/43a3b8964d994776817aa8528658714b/users/a7f1edf1fa6544079804e47c2c1939d2/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:22.214102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53dee2-04d2-43e0-906c-72f35de9743b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:22.214854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53dee2-04d2-43e0-906c-72f35de9743b admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:22.214938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff066c-2c46-477a-9167-133e162cee80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43a3b8964d994776817aa8528658714b, user_id=a7f1edf1fa6544079804e47c2c1939d2, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:22.215027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53dee2-04d2-43e0-906c-72f35de9743b admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:22.215332 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53dee2-04d2-43e0-906c-72f35de9743b admin admin] PATH_INFO: `/v3/projects/1088a8fb7f2f4544a00955c5ba7d2654/users/6ef33be1b1c44218b33931e8eb99b577/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:22.219310 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff066c-2c46-477a-9167-133e162cee80 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:22.219703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53dee2-04d2-43e0-906c-72f35de9743b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1088a8fb7f2f4544a00955c5ba7d2654, user_id=6ef33be1b1c44218b33931e8eb99b577, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:22.225537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53dee2-04d2-43e0-906c-72f35de9743b admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:22.247188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 818/1636] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:22 2022] PUT /identity/v3/projects/43a3b8964d994776817aa8528658714b/users/a7f1edf1fa6544079804e47c2c1939d2/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:22.251493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdc740-ec0a-44f9-939b-5b0fe696462c None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:22.251836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdc740-ec0a-44f9-939b-5b0fe696462c None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:22.252128 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdc740-ec0a-44f9-939b-5b0fe696462c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:22.262892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 819/1637] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:22 2022] PUT /identity/v3/projects/1088a8fb7f2f4544a00955c5ba7d2654/users/6ef33be1b1c44218b33931e8eb99b577/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:22.266245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8d23a-44fb-4daf-a688-c92ed50f292c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:22.266790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8d23a-44fb-4daf-a688-c92ed50f292c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:22.266913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8d23a-44fb-4daf-a688-c92ed50f292c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:22.336782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-71b8d23a-44fb-4daf-a688-c92ed50f292c None None] MFA Rules not processed for user `6ef33be1b1c44218b33931e8eb99b577`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:22.339326 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-71b8d23a-44fb-4daf-a688-c92ed50f292c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:22.346812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-58bdc740-ec0a-44f9-939b-5b0fe696462c None None] MFA Rules not processed for user `a7f1edf1fa6544079804e47c2c1939d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:22.351212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-58bdc740-ec0a-44f9-939b-5b0fe696462c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:22.472448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:22.524079 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 820/1638] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:16:22.541580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8d23a-44fb-4daf-a688-c92ed50f292c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:22.575133 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 819/1639] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:16:22.590566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdc740-ec0a-44f9-939b-5b0fe696462c None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:22.635705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:22.670939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8d23a-44fb-4daf-a688-c92ed50f292c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:22.675630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87c8c8-3741-47a5-b674-1fd2c807279a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:22.676280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87c8c8-3741-47a5-b674-1fd2c807279a service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:22.676564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87c8c8-3741-47a5-b674-1fd2c807279a service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:22.676831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87c8c8-3741-47a5-b674-1fd2c807279a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:22.688196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87c8c8-3741-47a5-b674-1fd2c807279a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:22.689736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87c8c8-3741-47a5-b674-1fd2c807279a service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:22.701095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 821/1640] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:22 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:22.747724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdc740-ec0a-44f9-939b-5b0fe696462c None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:22.747724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91b7a5-f404-4eef-9159-82fce5c1be1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:22.747724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91b7a5-f404-4eef-9159-82fce5c1be1a service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:22.760047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91b7a5-f404-4eef-9159-82fce5c1be1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:22.760047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91b7a5-f404-4eef-9159-82fce5c1be1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:22.765851 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91b7a5-f404-4eef-9159-82fce5c1be1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:22.765851 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91b7a5-f404-4eef-9159-82fce5c1be1a service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:22.789124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 820/1641] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:22.934907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87c8c8-3741-47a5-b674-1fd2c807279a service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:22.936842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91b7a5-f404-4eef-9159-82fce5c1be1a service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:22.944977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87c8c8-3741-47a5-b674-1fd2c807279a service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:22.947159 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91b7a5-f404-4eef-9159-82fce5c1be1a service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:22.956176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cf55a-3d85-4aa8-a3d9-38fa0d763c1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:22.956862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cf55a-3d85-4aa8-a3d9-38fa0d763c1d service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:22.957239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cf55a-3d85-4aa8-a3d9-38fa0d763c1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:22.957546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cf55a-3d85-4aa8-a3d9-38fa0d763c1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:22.962093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75424953-3baf-41dd-9a41-84572a41cee1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:22.963030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75424953-3baf-41dd-9a41-84572a41cee1 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:22.963598 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75424953-3baf-41dd-9a41-84572a41cee1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:22.963959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75424953-3baf-41dd-9a41-84572a41cee1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:23.129377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75424953-3baf-41dd-9a41-84572a41cee1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:23.135524 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75424953-3baf-41dd-9a41-84572a41cee1 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:23.158232 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cf55a-3d85-4aa8-a3d9-38fa0d763c1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:23.161236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 821/1642] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:23.176206 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cf55a-3d85-4aa8-a3d9-38fa0d763c1d service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:23.211221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 822/1643] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:23.494406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe635b8-f291-413b-b6a6-5c84eff92ffc None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:23.495040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe635b8-f291-413b-b6a6-5c84eff92ffc None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:23.495519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe635b8-f291-413b-b6a6-5c84eff92ffc None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:23.498435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 822/1644] 10.222.0.61 () {54 vars in 962 bytes} [Wed Sep 7 13:16:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:16:23.522084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb02d39-412a-43bf-8eb9-9f0bec6fe5d5 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:23.522381 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb02d39-412a-43bf-8eb9-9f0bec6fe5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:23.522660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb02d39-412a-43bf-8eb9-9f0bec6fe5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:23.592772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ffb02d39-412a-43bf-8eb9-9f0bec6fe5d5 None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:23.594639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ffb02d39-412a-43bf-8eb9-9f0bec6fe5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:23.796204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 823/1645] 10.222.0.61 () {58 vars in 1059 bytes} [Wed Sep 7 13:16:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:23.804962 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea682a9e-2a50-46f5-aa15-8b9279248dbf None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:23.805545 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea682a9e-2a50-46f5-aa15-8b9279248dbf None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:23.805967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea682a9e-2a50-46f5-aa15-8b9279248dbf None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:23.808105 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 823/1646] 10.222.0.61 () {54 vars in 912 bytes} [Wed Sep 7 13:16:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:16:23.814267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845e4ab-01fb-496e-a5b3-44bfefe330f1 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:23.814776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845e4ab-01fb-496e-a5b3-44bfefe330f1 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:23.817398 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845e4ab-01fb-496e-a5b3-44bfefe330f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:23.883192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f845e4ab-01fb-496e-a5b3-44bfefe330f1 None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:23.884484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f845e4ab-01fb-496e-a5b3-44bfefe330f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:23.922579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 824/1647] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:16:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:23.937084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea682a9e-2a50-46f5-aa15-8b9279248dbf None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:23.961000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea682a9e-2a50-46f5-aa15-8b9279248dbf None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:23.965359 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d450b-3b75-4cba-a800-4c162e3e9529 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:23.966000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d450b-3b75-4cba-a800-4c162e3e9529 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:23.966277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d450b-3b75-4cba-a800-4c162e3e9529 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:23.966740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d450b-3b75-4cba-a800-4c162e3e9529 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:23.992614 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d450b-3b75-4cba-a800-4c162e3e9529 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:23.994586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d450b-3b75-4cba-a800-4c162e3e9529 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:24.017571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 824/1648] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:24.251857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a636bc-c95e-4579-b180-28d7ce894e54 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:24.252393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a636bc-c95e-4579-b180-28d7ce894e54 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:24.252393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a636bc-c95e-4579-b180-28d7ce894e54 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:24.254219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 825/1649] 10.222.0.61 () {54 vars in 912 bytes} [Wed Sep 7 13:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:16:24.260479 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfddc95-4a41-4503-ab63-4bdb9b697caa None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:24.260762 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfddc95-4a41-4503-ab63-4bdb9b697caa None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:24.261566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfddc95-4a41-4503-ab63-4bdb9b697caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:24.304482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-7bfddc95-4a41-4503-ab63-4bdb9b697caa None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:24.306024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7bfddc95-4a41-4503-ab63-4bdb9b697caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:24.319476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 825/1650] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:16:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:24.338922 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a636bc-c95e-4579-b180-28d7ce894e54 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:24.352470 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a636bc-c95e-4579-b180-28d7ce894e54 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:24.360244 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93268e05-4e43-48aa-a181-5841c71ed426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:24.361653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93268e05-4e43-48aa-a181-5841c71ed426 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:24.362214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93268e05-4e43-48aa-a181-5841c71ed426 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:24.364112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93268e05-4e43-48aa-a181-5841c71ed426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:24.391605 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93268e05-4e43-48aa-a181-5841c71ed426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:24.394564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93268e05-4e43-48aa-a181-5841c71ed426 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:24.454841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 826/1651] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:24.586904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38351f-176a-42ec-bb51-fbe77fd4b477 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:24.588146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38351f-176a-42ec-bb51-fbe77fd4b477 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:24.588146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38351f-176a-42ec-bb51-fbe77fd4b477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:24.692634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-9d38351f-176a-42ec-bb51-fbe77fd4b477 None None] MFA Rules not processed for user `8f6db3f4d7df41c5be5a22a753c5606d`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:24.693942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9d38351f-176a-42ec-bb51-fbe77fd4b477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:24.718245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93268e05-4e43-48aa-a181-5841c71ed426 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:24.729858 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:24.883603 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93268e05-4e43-48aa-a181-5841c71ed426 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:24.895308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 826/1652] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:24 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:24.898342 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c29c8-7bcf-44fd-94c1-eafb59aa4c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:24.899082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c29c8-7bcf-44fd-94c1-eafb59aa4c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:24.899489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c29c8-7bcf-44fd-94c1-eafb59aa4c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:24.899861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c29c8-7bcf-44fd-94c1-eafb59aa4c82 admin admin] PATH_INFO: `/v3/users/cc2142713d524f4da090c4fc72b25db9` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:24.900680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38351f-176a-42ec-bb51-fbe77fd4b477 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:24.901130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c29c8-7bcf-44fd-94c1-eafb59aa4c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2142713d524f4da090c4fc72b25db9)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:24.904787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c29c8-7bcf-44fd-94c1-eafb59aa4c82 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:24.945992 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38351f-176a-42ec-bb51-fbe77fd4b477 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:24.965349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf532ad-7135-4557-ba23-0317c16f50b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:24.966239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf532ad-7135-4557-ba23-0317c16f50b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:24.967625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf532ad-7135-4557-ba23-0317c16f50b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:24.968104 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf532ad-7135-4557-ba23-0317c16f50b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:24.973684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf532ad-7135-4557-ba23-0317c16f50b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:24.981176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf532ad-7135-4557-ba23-0317c16f50b7 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:25.025804 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-bc2c29c8-7bcf-44fd-94c1-eafb59aa4c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2142713d524f4da090c4fc72b25db9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:25.044680 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-bc2c29c8-7bcf-44fd-94c1-eafb59aa4c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2142713d524f4da090c4fc72b25db9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:25.045504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-bc2c29c8-7bcf-44fd-94c1-eafb59aa4c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2142713d524f4da090c4fc72b25db9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:25.046402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-bc2c29c8-7bcf-44fd-94c1-eafb59aa4c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2142713d524f4da090c4fc72b25db9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:25.082097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-bc2c29c8-7bcf-44fd-94c1-eafb59aa4c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2142713d524f4da090c4fc72b25db9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:25.089191 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 827/1653] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:24 2022] POST /identity/v3/projects => generated 403 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:25.094845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf532ad-7135-4557-ba23-0317c16f50b7 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:25.103477 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-adf532ad-7135-4557-ba23-0317c16f50b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:25.123365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 827/1654] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:16:24 2022] DELETE /identity/v3/users/cc2142713d524f4da090c4fc72b25db9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:25.128060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:25.131486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ae9ec-4943-4c14-b2d1-795035bde25b None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:25.132420 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ae9ec-4943-4c14-b2d1-795035bde25b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:25.132789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ae9ec-4943-4c14-b2d1-795035bde25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:25.210501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-692ae9ec-4943-4c14-b2d1-795035bde25b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:25.218232 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-692ae9ec-4943-4c14-b2d1-795035bde25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:25.305091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf532ad-7135-4557-ba23-0317c16f50b7 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:25.456492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccaa8cd-faec-46f4-8c89-c10cdb7bb48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:25.457435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccaa8cd-faec-46f4-8c89-c10cdb7bb48c admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:25.457920 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccaa8cd-faec-46f4-8c89-c10cdb7bb48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:25.458363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccaa8cd-faec-46f4-8c89-c10cdb7bb48c admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:25.459233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccaa8cd-faec-46f4-8c89-c10cdb7bb48c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:25.461568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccaa8cd-faec-46f4-8c89-c10cdb7bb48c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:25.470297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 828/1655] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:25.533244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 828/1656] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:25 2022] POST /identity/v3/users => generated 308 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:25.533935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ae9ec-4943-4c14-b2d1-795035bde25b None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:25.554960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ae9ec-4943-4c14-b2d1-795035bde25b None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:25.569356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1982e4-f830-4952-b176-928604c141d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:25.569989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1982e4-f830-4952-b176-928604c141d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:25.570266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1982e4-f830-4952-b176-928604c141d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:25.570938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1982e4-f830-4952-b176-928604c141d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:25.571220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1982e4-f830-4952-b176-928604c141d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:25.572855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1982e4-f830-4952-b176-928604c141d9 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:25.614668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 829/1657] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:25.619480 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccaa8cd-faec-46f4-8c89-c10cdb7bb48c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:25.636204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccaa8cd-faec-46f4-8c89-c10cdb7bb48c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:25.646623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a5be1-7d72-4bef-87f1-b652fa961810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:25.648080 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a5be1-7d72-4bef-87f1-b652fa961810 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:25.648539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a5be1-7d72-4bef-87f1-b652fa961810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:25.649474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a5be1-7d72-4bef-87f1-b652fa961810 admin admin] PATH_INFO: `/v3/projects/15b14c7b75fe4933a275eb9d97c6ce55/users/7da095d8493f4e14895a6ddeaf0b3b81/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:25.675284 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1982e4-f830-4952-b176-928604c141d9 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:25.680763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3f1982e4-f830-4952-b176-928604c141d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:25.724093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:25.742652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a5be1-7d72-4bef-87f1-b652fa961810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b14c7b75fe4933a275eb9d97c6ce55, user_id=7da095d8493f4e14895a6ddeaf0b3b81, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:25.750931 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a5be1-7d72-4bef-87f1-b652fa961810 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:25.805950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 829/1658] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:25 2022] PUT /identity/v3/projects/15b14c7b75fe4933a275eb9d97c6ce55/users/7da095d8493f4e14895a6ddeaf0b3b81/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:25.809967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a5be1-7d72-4bef-87f1-b652fa961810 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:25.929034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1982e4-f830-4952-b176-928604c141d9 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:26.073494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a5be1-7d72-4bef-87f1-b652fa961810 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:26.100307 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83c05a-d3ed-485d-8f9b-48ac65457c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:26.101059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83c05a-d3ed-485d-8f9b-48ac65457c97 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:26.101329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83c05a-d3ed-485d-8f9b-48ac65457c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:26.101671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83c05a-d3ed-485d-8f9b-48ac65457c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:26.102278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83c05a-d3ed-485d-8f9b-48ac65457c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:26.107126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83c05a-d3ed-485d-8f9b-48ac65457c97 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:26.116187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d80a6-76de-4130-b4b9-bc01482e5fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:26.116831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d80a6-76de-4130-b4b9-bc01482e5fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:26.117108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d80a6-76de-4130-b4b9-bc01482e5fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:26.117375 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d80a6-76de-4130-b4b9-bc01482e5fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:26.153115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 830/1659] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:26.155018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83c05a-d3ed-485d-8f9b-48ac65457c97 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:26.158274 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d80a6-76de-4130-b4b9-bc01482e5fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:26.164451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d80a6-76de-4130-b4b9-bc01482e5fc0 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:26.189089 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83c05a-d3ed-485d-8f9b-48ac65457c97 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:26.204187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 830/1660] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:16:26.210675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e069686-3dec-4dc0-ae63-296e6485e111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:26.211406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e069686-3dec-4dc0-ae63-296e6485e111 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:26.211687 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e069686-3dec-4dc0-ae63-296e6485e111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:26.211956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e069686-3dec-4dc0-ae63-296e6485e111 admin admin] PATH_INFO: `/v3/projects/15b14c7b75fe4933a275eb9d97c6ce55/users/7da095d8493f4e14895a6ddeaf0b3b81/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:26.218472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e069686-3dec-4dc0-ae63-296e6485e111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b14c7b75fe4933a275eb9d97c6ce55, user_id=7da095d8493f4e14895a6ddeaf0b3b81, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:26.221262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e069686-3dec-4dc0-ae63-296e6485e111 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:26.273069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 831/1661] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:26 2022] PUT /identity/v3/projects/15b14c7b75fe4933a275eb9d97c6ce55/users/7da095d8493f4e14895a6ddeaf0b3b81/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:29.228506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe43b61-6280-4e0e-a75c-08a519b3284c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:29.229753 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe43b61-6280-4e0e-a75c-08a519b3284c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:29.230049 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe43b61-6280-4e0e-a75c-08a519b3284c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:29.289925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ebe43b61-6280-4e0e-a75c-08a519b3284c None None] MFA Rules not processed for user `181c888fbd454f00b049522fdfb4197e`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:29.291780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ebe43b61-6280-4e0e-a75c-08a519b3284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:29.398706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 831/1662] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:29.416711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e069686-3dec-4dc0-ae63-296e6485e111 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:29.429166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3e069686-3dec-4dc0-ae63-296e6485e111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:29.445817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:29.596862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e069686-3dec-4dc0-ae63-296e6485e111 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:29.604862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc652f0-7998-4872-8eda-e4d197d8813c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:29.606082 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc652f0-7998-4872-8eda-e4d197d8813c service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:29.606578 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc652f0-7998-4872-8eda-e4d197d8813c service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:29.607733 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc652f0-7998-4872-8eda-e4d197d8813c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:29.630193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc652f0-7998-4872-8eda-e4d197d8813c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:29.632723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc652f0-7998-4872-8eda-e4d197d8813c service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:29.655642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 832/1663] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:29.896760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe43b61-6280-4e0e-a75c-08a519b3284c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:29.899525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ebe43b61-6280-4e0e-a75c-08a519b3284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:30.153748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe43b61-6280-4e0e-a75c-08a519b3284c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:30.170317 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8646c6-19d7-4c44-92a7-ad36e3d01874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:30.171286 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8646c6-19d7-4c44-92a7-ad36e3d01874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:30.172064 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8646c6-19d7-4c44-92a7-ad36e3d01874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:30.172556 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8646c6-19d7-4c44-92a7-ad36e3d01874 admin admin] PATH_INFO: `/v3/projects/267fd08403b049f0b6464921bc8fa2f3` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:30.174260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8646c6-19d7-4c44-92a7-ad36e3d01874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=267fd08403b049f0b6464921bc8fa2f3)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:30.176012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: WARNING py.warnings [None req-9e8646c6-19d7-4c44-92a7-ad36e3d01874 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:16:30.176012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: warnings.warn(msg) Sep 07 13:16:30.176012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: Sep 07 13:16:30.177724 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8646c6-19d7-4c44-92a7-ad36e3d01874 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:30.183826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:30.350364 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9e8646c6-19d7-4c44-92a7-ad36e3d01874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '267fd08403b049f0b6464921bc8fa2f3', 'request_id': 'req-9e8646c6-19d7-4c44-92a7-ad36e3d01874'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:30.373856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9e8646c6-19d7-4c44-92a7-ad36e3d01874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '267fd08403b049f0b6464921bc8fa2f3', 'request_id': 'req-9e8646c6-19d7-4c44-92a7-ad36e3d01874'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:30.389526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9e8646c6-19d7-4c44-92a7-ad36e3d01874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '267fd08403b049f0b6464921bc8fa2f3', 'request_id': 'req-9e8646c6-19d7-4c44-92a7-ad36e3d01874'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:30.438532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9e8646c6-19d7-4c44-92a7-ad36e3d01874 admin admin] The token cache is being invalidate because project 267fd08403b049f0b6464921bc8fa2f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:16:30.439632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9e8646c6-19d7-4c44-92a7-ad36e3d01874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:30.440756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9e8646c6-19d7-4c44-92a7-ad36e3d01874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:30.442648 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 832/1664] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:16:29 2022] DELETE /identity/v3/projects/267fd08403b049f0b6464921bc8fa2f3 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:30.448428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86331547-4402-43e3-bb45-9f7a2c7b4082 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:30.448953 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86331547-4402-43e3-bb45-9f7a2c7b4082 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:30.448953 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86331547-4402-43e3-bb45-9f7a2c7b4082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:30.495589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-86331547-4402-43e3-bb45-9f7a2c7b4082 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:30.509956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-86331547-4402-43e3-bb45-9f7a2c7b4082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:30.632949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 833/1665] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:30.644856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3a1a0-5aac-4cbd-b987-691179fa771f None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:30.645140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3a1a0-5aac-4cbd-b987-691179fa771f None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:30.645600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3a1a0-5aac-4cbd-b987-691179fa771f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:30.701855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-5eb3a1a0-5aac-4cbd-b987-691179fa771f None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:30.703207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5eb3a1a0-5aac-4cbd-b987-691179fa771f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:30.731589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86331547-4402-43e3-bb45-9f7a2c7b4082 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:30.733480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 833/1666] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:30.746333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86331547-4402-43e3-bb45-9f7a2c7b4082 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:30.754176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420fa48-6036-4740-a8de-30287843a37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:30.754888 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420fa48-6036-4740-a8de-30287843a37e admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:30.755276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420fa48-6036-4740-a8de-30287843a37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:30.755700 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420fa48-6036-4740-a8de-30287843a37e admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:30.757178 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420fa48-6036-4740-a8de-30287843a37e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:16:30.757561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420fa48-6036-4740-a8de-30287843a37e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:30.760486 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420fa48-6036-4740-a8de-30287843a37e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:30.789559 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3a1a0-5aac-4cbd-b987-691179fa771f None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:30.797558 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 834/1667] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:16:30.814672 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3a1a0-5aac-4cbd-b987-691179fa771f None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:30.831903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479acf38-e20c-4117-afa5-a20f08de49d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:30.833895 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479acf38-e20c-4117-afa5-a20f08de49d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:30.834443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479acf38-e20c-4117-afa5-a20f08de49d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:30.834914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479acf38-e20c-4117-afa5-a20f08de49d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:30.835820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479acf38-e20c-4117-afa5-a20f08de49d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:30.838444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479acf38-e20c-4117-afa5-a20f08de49d3 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:30.892965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 834/1668] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:30 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:30.897444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420fa48-6036-4740-a8de-30287843a37e admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:30.929386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420fa48-6036-4740-a8de-30287843a37e admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:31.093262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab1a1f-b7bc-4fc1-92a7-353d7e71abcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:31.094338 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab1a1f-b7bc-4fc1-92a7-353d7e71abcc admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:31.094787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab1a1f-b7bc-4fc1-92a7-353d7e71abcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:31.095259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab1a1f-b7bc-4fc1-92a7-353d7e71abcc admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:31.096666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab1a1f-b7bc-4fc1-92a7-353d7e71abcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:31.105666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab1a1f-b7bc-4fc1-92a7-353d7e71abcc admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:31.151042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 835/1669] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:30 2022] POST /identity/v3/users => generated 282 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:16:31.155057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479acf38-e20c-4117-afa5-a20f08de49d3 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:31.170929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479acf38-e20c-4117-afa5-a20f08de49d3 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:31.177189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee49338-02ba-474b-ab8d-42f064a16411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:31.177884 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee49338-02ba-474b-ab8d-42f064a16411 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:31.178227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee49338-02ba-474b-ab8d-42f064a16411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:31.178653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee49338-02ba-474b-ab8d-42f064a16411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:31.179525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee49338-02ba-474b-ab8d-42f064a16411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:31.181372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee49338-02ba-474b-ab8d-42f064a16411 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:31.202677 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 835/1670] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:16:31.204892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab1a1f-b7bc-4fc1-92a7-353d7e71abcc admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:31.224714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:31.232981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab1a1f-b7bc-4fc1-92a7-353d7e71abcc admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:31.241206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff743a1-fd53-4d2d-801a-5e036f1d6466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:31.242167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff743a1-fd53-4d2d-801a-5e036f1d6466 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:31.242625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff743a1-fd53-4d2d-801a-5e036f1d6466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:31.243074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff743a1-fd53-4d2d-801a-5e036f1d6466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:31.244132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff743a1-fd53-4d2d-801a-5e036f1d6466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:31.245978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff743a1-fd53-4d2d-801a-5e036f1d6466 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:31.292822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee49338-02ba-474b-ab8d-42f064a16411 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:31.293068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 836/1671] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:31.319876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee49338-02ba-474b-ab8d-42f064a16411 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:31.329479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb62d3c-36a0-4d1c-a8b6-123bc44486c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:31.343715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb62d3c-36a0-4d1c-a8b6-123bc44486c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:31.344578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb62d3c-36a0-4d1c-a8b6-123bc44486c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:31.345463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb62d3c-36a0-4d1c-a8b6-123bc44486c9 admin admin] PATH_INFO: `/v3/projects/cf2bc6b24c4b46eda718a42805828757/users/ac79236e8106438ca288beff5b0e44a1/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:31.371197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb62d3c-36a0-4d1c-a8b6-123bc44486c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf2bc6b24c4b46eda718a42805828757, user_id=ac79236e8106438ca288beff5b0e44a1, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:31.374003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb62d3c-36a0-4d1c-a8b6-123bc44486c9 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:31.392666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 836/1672] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:31 2022] PUT /identity/v3/projects/cf2bc6b24c4b46eda718a42805828757/users/ac79236e8106438ca288beff5b0e44a1/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:31.405767 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e4978-0e06-4392-ab78-c1fa2ff73a30 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:31.406190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e4978-0e06-4392-ab78-c1fa2ff73a30 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:31.406328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e4978-0e06-4392-ab78-c1fa2ff73a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:31.475046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-5a6e4978-0e06-4392-ab78-c1fa2ff73a30 None None] MFA Rules not processed for user `ac79236e8106438ca288beff5b0e44a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:31.478507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5a6e4978-0e06-4392-ab78-c1fa2ff73a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:31.671152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 837/1673] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:31.683073 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb62d3c-36a0-4d1c-a8b6-123bc44486c9 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:31.812117 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb62d3c-36a0-4d1c-a8b6-123bc44486c9 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:31.818915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a28f7-2ec0-4ebf-9813-02e89e9d50ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:31.819689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a28f7-2ec0-4ebf-9813-02e89e9d50ca admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:31.820047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a28f7-2ec0-4ebf-9813-02e89e9d50ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:31.820387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a28f7-2ec0-4ebf-9813-02e89e9d50ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:31.822072 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a28f7-2ec0-4ebf-9813-02e89e9d50ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:31.825604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a28f7-2ec0-4ebf-9813-02e89e9d50ca admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:31.873166 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 837/1674] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:31 2022] POST /identity/v3/projects => generated 361 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:31.877381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e4978-0e06-4392-ab78-c1fa2ff73a30 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:31.904805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e4978-0e06-4392-ab78-c1fa2ff73a30 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:32.040116 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96b6e2-2900-4fb2-921d-324615af93b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:32.040789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96b6e2-2900-4fb2-921d-324615af93b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:32.041060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96b6e2-2900-4fb2-921d-324615af93b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:32.041699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96b6e2-2900-4fb2-921d-324615af93b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:32.042177 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96b6e2-2900-4fb2-921d-324615af93b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:32.045161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96b6e2-2900-4fb2-921d-324615af93b3 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:32.090580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f48ffa-8d82-49fb-afd4-43879f2ce110 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:32.090580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f48ffa-8d82-49fb-afd4-43879f2ce110 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:32.090580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f48ffa-8d82-49fb-afd4-43879f2ce110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:32.115325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 838/1675] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:31 2022] POST /identity/v3/users => generated 287 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:32.119295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96b6e2-2900-4fb2-921d-324615af93b3 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:32.157195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96b6e2-2900-4fb2-921d-324615af93b3 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:32.182051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:32.185215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052f1ec-0b4c-4f3d-98e5-0a205cb082ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:32.185868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052f1ec-0b4c-4f3d-98e5-0a205cb082ce admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:32.186156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052f1ec-0b4c-4f3d-98e5-0a205cb082ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:32.186755 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052f1ec-0b4c-4f3d-98e5-0a205cb082ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:32.187283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052f1ec-0b4c-4f3d-98e5-0a205cb082ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:32.188961 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052f1ec-0b4c-4f3d-98e5-0a205cb082ce admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:32.199757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-46f48ffa-8d82-49fb-afd4-43879f2ce110 None None] MFA Rules not processed for user `7da095d8493f4e14895a6ddeaf0b3b81`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:32.203131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-46f48ffa-8d82-49fb-afd4-43879f2ce110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:32.237540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 839/1676] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:32.238410 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052f1ec-0b4c-4f3d-98e5-0a205cb082ce admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:32.267749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052f1ec-0b4c-4f3d-98e5-0a205cb082ce admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:32.304103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123ec3e-33cf-43b7-947e-dcd2e2cbbb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:32.304804 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123ec3e-33cf-43b7-947e-dcd2e2cbbb62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:32.305088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123ec3e-33cf-43b7-947e-dcd2e2cbbb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:32.305369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123ec3e-33cf-43b7-947e-dcd2e2cbbb62 admin admin] PATH_INFO: `/v3/projects/c8df7cad7caf4de5ac2b27970319ce94/users/7276f5883af54398b3642545110b1adc/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:32.348582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123ec3e-33cf-43b7-947e-dcd2e2cbbb62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8df7cad7caf4de5ac2b27970319ce94, user_id=7276f5883af54398b3642545110b1adc, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:32.353530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123ec3e-33cf-43b7-947e-dcd2e2cbbb62 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:32.382419 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 840/1677] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:32 2022] PUT /identity/v3/projects/c8df7cad7caf4de5ac2b27970319ce94/users/7276f5883af54398b3642545110b1adc/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:32.387617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123ec3e-33cf-43b7-947e-dcd2e2cbbb62 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:32.426715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 838/1678] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:32 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:32.495751 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:32.539064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123ec3e-33cf-43b7-947e-dcd2e2cbbb62 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:32.549438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21465d91-6476-4cc4-839a-cc8666f6a524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:32.550401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21465d91-6476-4cc4-839a-cc8666f6a524 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:32.550846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21465d91-6476-4cc4-839a-cc8666f6a524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:32.551287 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21465d91-6476-4cc4-839a-cc8666f6a524 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:32.552119 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21465d91-6476-4cc4-839a-cc8666f6a524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:32.554041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21465d91-6476-4cc4-839a-cc8666f6a524 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:32.580725 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 841/1679] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:16:32.582374 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f48ffa-8d82-49fb-afd4-43879f2ce110 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:32.598877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f48ffa-8d82-49fb-afd4-43879f2ce110 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:32.616036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e644b-2cac-483b-87ff-7d833a5c4762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:32.616999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e644b-2cac-483b-87ff-7d833a5c4762 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:32.617462 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e644b-2cac-483b-87ff-7d833a5c4762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:32.617884 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e644b-2cac-483b-87ff-7d833a5c4762 admin admin] PATH_INFO: `/v3/projects/c8df7cad7caf4de5ac2b27970319ce94/users/7276f5883af54398b3642545110b1adc/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:32.628377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e644b-2cac-483b-87ff-7d833a5c4762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8df7cad7caf4de5ac2b27970319ce94, user_id=7276f5883af54398b3642545110b1adc, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:32.632120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e644b-2cac-483b-87ff-7d833a5c4762 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:32.657905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 839/1680] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:32 2022] PUT /identity/v3/projects/c8df7cad7caf4de5ac2b27970319ce94/users/7276f5883af54398b3642545110b1adc/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:32.669515 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a41946-690b-4a61-b2f4-78c1d4dd8189 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:32.670048 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a41946-690b-4a61-b2f4-78c1d4dd8189 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:32.670472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a41946-690b-4a61-b2f4-78c1d4dd8189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:32.747145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-83a41946-690b-4a61-b2f4-78c1d4dd8189 None None] MFA Rules not processed for user `7276f5883af54398b3642545110b1adc`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:32.750050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-83a41946-690b-4a61-b2f4-78c1d4dd8189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:32.949272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 842/1681] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:32 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:32.957507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e644b-2cac-483b-87ff-7d833a5c4762 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:32.959366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ec8e644b-2cac-483b-87ff-7d833a5c4762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:33.064860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e644b-2cac-483b-87ff-7d833a5c4762 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:33.080426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b387581-99a5-40ee-bdbc-b11627db449b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:33.082020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b387581-99a5-40ee-bdbc-b11627db449b service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:33.082998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b387581-99a5-40ee-bdbc-b11627db449b service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:33.083802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b387581-99a5-40ee-bdbc-b11627db449b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:33.105996 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b387581-99a5-40ee-bdbc-b11627db449b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:33.110131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b387581-99a5-40ee-bdbc-b11627db449b service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:33.146983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 840/1682] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:32 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:16:33.398076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a41946-690b-4a61-b2f4-78c1d4dd8189 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:33.401348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-83a41946-690b-4a61-b2f4-78c1d4dd8189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:33.444951 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a41946-690b-4a61-b2f4-78c1d4dd8189 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:33.461631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd231245-746b-45df-9e61-43ebb7261b2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:33.463304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd231245-746b-45df-9e61-43ebb7261b2c service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:33.463882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd231245-746b-45df-9e61-43ebb7261b2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:33.464502 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd231245-746b-45df-9e61-43ebb7261b2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:33.648347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd231245-746b-45df-9e61-43ebb7261b2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:33.650403 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd231245-746b-45df-9e61-43ebb7261b2c service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:33.664882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 843/1683] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:34.948652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f54abe-fc45-425b-b8c2-bb09f85cf152 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:34.949550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f54abe-fc45-425b-b8c2-bb09f85cf152 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:34.949973 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f54abe-fc45-425b-b8c2-bb09f85cf152 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:34.952448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 841/1684] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:16:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:16:34.975873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf37e7-c552-479d-b589-ee05826fb334 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:34.976732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf37e7-c552-479d-b589-ee05826fb334 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:34.977237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf37e7-c552-479d-b589-ee05826fb334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:35.016859 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-29bf37e7-c552-479d-b589-ee05826fb334 None None] MFA Rules not processed for user `9e283e62741f40439d02e545a212807a`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:35.021362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-29bf37e7-c552-479d-b589-ee05826fb334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:35.196443 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:35.239506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 844/1685] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:16:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:16:35.288065 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f54abe-fc45-425b-b8c2-bb09f85cf152 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:35.290941 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-03f54abe-fc45-425b-b8c2-bb09f85cf152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:35.323632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f54abe-fc45-425b-b8c2-bb09f85cf152 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:35.331890 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8374832-fb9c-4e66-bb03-64d130a44856 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:35.333001 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8374832-fb9c-4e66-bb03-64d130a44856 service placement] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:35.333468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8374832-fb9c-4e66-bb03-64d130a44856 service placement] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:35.333924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8374832-fb9c-4e66-bb03-64d130a44856 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:35.346823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8374832-fb9c-4e66-bb03-64d130a44856 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:35.348895 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8374832-fb9c-4e66-bb03-64d130a44856 service placement] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:35.376090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 842/1686] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:38.158490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16202af0-ebd2-483c-96a1-9dd364aabdd5 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:38.160163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16202af0-ebd2-483c-96a1-9dd364aabdd5 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:38.160608 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16202af0-ebd2-483c-96a1-9dd364aabdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:38.289880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-16202af0-ebd2-483c-96a1-9dd364aabdd5 None None] MFA Rules not processed for user `cf718a19fbc9458c996678ed7accc344`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:38.291147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-16202af0-ebd2-483c-96a1-9dd364aabdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:38.440291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 845/1687] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:38.457825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d256a-be90-4959-a860-e0cd81d30f88 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:38.458467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d256a-be90-4959-a860-e0cd81d30f88 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:38.458467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d256a-be90-4959-a860-e0cd81d30f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:38.492341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-b31d256a-be90-4959-a860-e0cd81d30f88 None None] MFA Rules not processed for user `cf718a19fbc9458c996678ed7accc344`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:38.496749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b31d256a-be90-4959-a860-e0cd81d30f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:38.515686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 843/1688] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:38.532769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16202af0-ebd2-483c-96a1-9dd364aabdd5 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:38.534811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-16202af0-ebd2-483c-96a1-9dd364aabdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:38.615542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57839ed3-1165-4376-9564-3e2619e5ec9f None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:38.615829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57839ed3-1165-4376-9564-3e2619e5ec9f None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:38.616268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57839ed3-1165-4376-9564-3e2619e5ec9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:38.657936 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-57839ed3-1165-4376-9564-3e2619e5ec9f None None] MFA Rules not processed for user `181c888fbd454f00b049522fdfb4197e`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:38.659420 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-57839ed3-1165-4376-9564-3e2619e5ec9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:38.787328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16202af0-ebd2-483c-96a1-9dd364aabdd5 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:38.819692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015edf92-b020-40e9-b576-3a8aa2c48c20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:38.822548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015edf92-b020-40e9-b576-3a8aa2c48c20 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:38.827146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015edf92-b020-40e9-b576-3a8aa2c48c20 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:38.827478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015edf92-b020-40e9-b576-3a8aa2c48c20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:38.829097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 844/1689] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:38.912013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015edf92-b020-40e9-b576-3a8aa2c48c20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:38.930902 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015edf92-b020-40e9-b576-3a8aa2c48c20 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:38.992835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 846/1690] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:40.002677 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57839ed3-1165-4376-9564-3e2619e5ec9f None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:40.005168 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-57839ed3-1165-4376-9564-3e2619e5ec9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:40.067058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:40.176504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015edf92-b020-40e9-b576-3a8aa2c48c20 service nova] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:40.238012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57839ed3-1165-4376-9564-3e2619e5ec9f None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:40.252638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838c83d-95ad-46a1-8ff9-a64b49e89270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:40.253589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838c83d-95ad-46a1-8ff9-a64b49e89270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:40.254062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838c83d-95ad-46a1-8ff9-a64b49e89270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:40.254537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838c83d-95ad-46a1-8ff9-a64b49e89270 admin admin] PATH_INFO: `/v3/users/8f6db3f4d7df41c5be5a22a753c5606d` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:40.256015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838c83d-95ad-46a1-8ff9-a64b49e89270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f6db3f4d7df41c5be5a22a753c5606d)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:40.258792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838c83d-95ad-46a1-8ff9-a64b49e89270 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:40.358506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6838c83d-95ad-46a1-8ff9-a64b49e89270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f6db3f4d7df41c5be5a22a753c5606d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:40.363893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015edf92-b020-40e9-b576-3a8aa2c48c20 service nova] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:40.374796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6838c83d-95ad-46a1-8ff9-a64b49e89270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f6db3f4d7df41c5be5a22a753c5606d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:40.377169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6838c83d-95ad-46a1-8ff9-a64b49e89270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f6db3f4d7df41c5be5a22a753c5606d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:40.378173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6838c83d-95ad-46a1-8ff9-a64b49e89270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f6db3f4d7df41c5be5a22a753c5606d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:40.382909 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76105b28-230a-449c-99bb-36e761420381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:40.383699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76105b28-230a-449c-99bb-36e761420381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:40.383989 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76105b28-230a-449c-99bb-36e761420381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:40.384267 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76105b28-230a-449c-99bb-36e761420381 admin admin] PATH_INFO: `/v3/users/ac79236e8106438ca288beff5b0e44a1` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:40.387881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76105b28-230a-449c-99bb-36e761420381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac79236e8106438ca288beff5b0e44a1)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:40.391570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: WARNING py.warnings [None req-76105b28-230a-449c-99bb-36e761420381 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:16:40.391570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: warnings.warn(msg) Sep 07 13:16:40.391570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: Sep 07 13:16:40.395775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76105b28-230a-449c-99bb-36e761420381 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:40.398160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6838c83d-95ad-46a1-8ff9-a64b49e89270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f6db3f4d7df41c5be5a22a753c5606d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:40.436087 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:40.439336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 845/1691] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:16:40 2022] DELETE /identity/v3/users/8f6db3f4d7df41c5be5a22a753c5606d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:40.554531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-76105b28-230a-449c-99bb-36e761420381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac79236e8106438ca288beff5b0e44a1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:40.588922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-76105b28-230a-449c-99bb-36e761420381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac79236e8106438ca288beff5b0e44a1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:40.610864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-76105b28-230a-449c-99bb-36e761420381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac79236e8106438ca288beff5b0e44a1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:40.612163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-76105b28-230a-449c-99bb-36e761420381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac79236e8106438ca288beff5b0e44a1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:40.613256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-76105b28-230a-449c-99bb-36e761420381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac79236e8106438ca288beff5b0e44a1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:40.639719 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 847/1692] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:16:40 2022] DELETE /identity/v3/users/ac79236e8106438ca288beff5b0e44a1 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:40.644423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbc4bb-45d3-4287-96e0-465471ec3396 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:40.644712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbc4bb-45d3-4287-96e0-465471ec3396 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:40.644990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbc4bb-45d3-4287-96e0-465471ec3396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:40.694772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f7cbc4bb-45d3-4287-96e0-465471ec3396 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:40.701423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f7cbc4bb-45d3-4287-96e0-465471ec3396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:40.742420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac670e86-4d63-4aac-9878-efc73b017829 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:40.742706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac670e86-4d63-4aac-9878-efc73b017829 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:40.742979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac670e86-4d63-4aac-9878-efc73b017829 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:40.744592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 848/1693] 10.222.0.61 () {54 vars in 912 bytes} [Wed Sep 7 13:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:16:40.750449 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00349bbf-67c1-44cf-8abb-d3575b138c35 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:40.750731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00349bbf-67c1-44cf-8abb-d3575b138c35 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:40.751003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00349bbf-67c1-44cf-8abb-d3575b138c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:40.864017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-00349bbf-67c1-44cf-8abb-d3575b138c35 None None] MFA Rules not processed for user `de05c08e30dd4fff93f1c97aac9553d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:40.874923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-00349bbf-67c1-44cf-8abb-d3575b138c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:40.911216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 846/1694] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:40.935731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbc4bb-45d3-4287-96e0-465471ec3396 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:40.937500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f7cbc4bb-45d3-4287-96e0-465471ec3396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:40.998374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 849/1695] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:16:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:16:41.029287 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00349bbf-67c1-44cf-8abb-d3575b138c35 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:41.030949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-00349bbf-67c1-44cf-8abb-d3575b138c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:41.051070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00349bbf-67c1-44cf-8abb-d3575b138c35 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:41.058110 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:41.073400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea4e72-8a4a-460a-95b1-efc4f6364f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:41.074134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea4e72-8a4a-460a-95b1-efc4f6364f90 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:41.075068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea4e72-8a4a-460a-95b1-efc4f6364f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:41.075068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea4e72-8a4a-460a-95b1-efc4f6364f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:41.093740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea4e72-8a4a-460a-95b1-efc4f6364f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:41.097597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea4e72-8a4a-460a-95b1-efc4f6364f90 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:41.118029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbc4bb-45d3-4287-96e0-465471ec3396 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:41.125675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 850/1696] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:16:41.135158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f077063-115c-46c4-95df-bb587e75cca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:41.135889 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f077063-115c-46c4-95df-bb587e75cca4 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:41.136316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f077063-115c-46c4-95df-bb587e75cca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:41.136596 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f077063-115c-46c4-95df-bb587e75cca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:41.164210 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f077063-115c-46c4-95df-bb587e75cca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:41.165532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f077063-115c-46c4-95df-bb587e75cca4 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:41.201991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 847/1697] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:16:41.282527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea4e72-8a4a-460a-95b1-efc4f6364f90 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:41.284550 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-08ea4e72-8a4a-460a-95b1-efc4f6364f90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:41.324838 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea4e72-8a4a-460a-95b1-efc4f6364f90 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:41.332014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f8396-2c77-49ac-865a-c88d98ac3967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:41.332969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f8396-2c77-49ac-865a-c88d98ac3967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:41.333387 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f8396-2c77-49ac-865a-c88d98ac3967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:41.333832 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f8396-2c77-49ac-865a-c88d98ac3967 admin admin] PATH_INFO: `/v3/projects/06e87ad5726745d5a06276f0eb82788a` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:41.335280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f8396-2c77-49ac-865a-c88d98ac3967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e87ad5726745d5a06276f0eb82788a)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:41.353393 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f8396-2c77-49ac-865a-c88d98ac3967 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:41.546828 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4e4f8396-2c77-49ac-865a-c88d98ac3967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e87ad5726745d5a06276f0eb82788a', 'request_id': 'req-4e4f8396-2c77-49ac-865a-c88d98ac3967'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:41.557982 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4e4f8396-2c77-49ac-865a-c88d98ac3967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e87ad5726745d5a06276f0eb82788a', 'request_id': 'req-4e4f8396-2c77-49ac-865a-c88d98ac3967'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:41.568838 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4e4f8396-2c77-49ac-865a-c88d98ac3967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e87ad5726745d5a06276f0eb82788a', 'request_id': 'req-4e4f8396-2c77-49ac-865a-c88d98ac3967'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:41.573798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:41.611562 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4e4f8396-2c77-49ac-865a-c88d98ac3967 admin admin] The token cache is being invalidate because project 06e87ad5726745d5a06276f0eb82788a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:16:41.612189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4e4f8396-2c77-49ac-865a-c88d98ac3967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:41.616666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4e4f8396-2c77-49ac-865a-c88d98ac3967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:41.619284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 851/1698] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:16:41 2022] DELETE /identity/v3/projects/06e87ad5726745d5a06276f0eb82788a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:41.623499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f077063-115c-46c4-95df-bb587e75cca4 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:41.641720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f077063-115c-46c4-95df-bb587e75cca4 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:41.642513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39b57d-002a-42e9-92a0-f4d6c251a1b0 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:41.643030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39b57d-002a-42e9-92a0-f4d6c251a1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:41.643789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39b57d-002a-42e9-92a0-f4d6c251a1b0 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:41.645412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 852/1699] 10.222.0.61 () {54 vars in 912 bytes} [Wed Sep 7 13:16:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:16:41.665950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b189d3-7821-45a6-af29-1d984fc10ac9 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:41.666490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b189d3-7821-45a6-af29-1d984fc10ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:41.666909 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b189d3-7821-45a6-af29-1d984fc10ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:41.751019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-36b189d3-7821-45a6-af29-1d984fc10ac9 None None] MFA Rules not processed for user `de05c08e30dd4fff93f1c97aac9553d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:41.755572 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-36b189d3-7821-45a6-af29-1d984fc10ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:41.907029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:41.907989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:41.909135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:41.909580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] PATH_INFO: `/v3/users/7da095d8493f4e14895a6ddeaf0b3b81` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:41.911244 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7da095d8493f4e14895a6ddeaf0b3b81)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:41.913651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:41.963930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 853/1700] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:16:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:41.966328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b189d3-7821-45a6-af29-1d984fc10ac9 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:41.971733 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-36b189d3-7821-45a6-af29-1d984fc10ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:42.020932 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7da095d8493f4e14895a6ddeaf0b3b81'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:42.039002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7da095d8493f4e14895a6ddeaf0b3b81'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:42.040085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7da095d8493f4e14895a6ddeaf0b3b81'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:42.041390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7da095d8493f4e14895a6ddeaf0b3b81'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:42.074967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7da095d8493f4e14895a6ddeaf0b3b81'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:42.079296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b189d3-7821-45a6-af29-1d984fc10ac9 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:42.120742 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 848/1701] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:16:41 2022] DELETE /identity/v3/users/7da095d8493f4e14895a6ddeaf0b3b81 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:42.121749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:42.123989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:42.129040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb4561-b8c2-457b-a5e2-53bf1a91c6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:42.136934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb4561-b8c2-457b-a5e2-53bf1a91c6d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:42.145390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb4561-b8c2-457b-a5e2-53bf1a91c6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:42.146291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb4561-b8c2-457b-a5e2-53bf1a91c6d0 admin admin] PATH_INFO: `/v3/users/5f8f8d47ecd646dca1362021e70657b6` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:42.149055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb4561-b8c2-457b-a5e2-53bf1a91c6d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8f8d47ecd646dca1362021e70657b6)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:42.153991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb4561-b8c2-457b-a5e2-53bf1a91c6d0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:42.329111 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5b814-3265-41df-89a6-1ab59497d76d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:42.340141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4abb4561-b8c2-457b-a5e2-53bf1a91c6d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8f8d47ecd646dca1362021e70657b6'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:42.351240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3a862-de18-4b9b-baa6-62ba2e493a16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:42.352007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3a862-de18-4b9b-baa6-62ba2e493a16 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:42.352917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3a862-de18-4b9b-baa6-62ba2e493a16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:42.354765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3a862-de18-4b9b-baa6-62ba2e493a16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:42.364627 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-83e3a862-de18-4b9b-baa6-62ba2e493a16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:42.367846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4abb4561-b8c2-457b-a5e2-53bf1a91c6d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8f8d47ecd646dca1362021e70657b6'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:42.384393 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4abb4561-b8c2-457b-a5e2-53bf1a91c6d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8f8d47ecd646dca1362021e70657b6'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:42.385540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4abb4561-b8c2-457b-a5e2-53bf1a91c6d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8f8d47ecd646dca1362021e70657b6'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:42.386809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4abb4561-b8c2-457b-a5e2-53bf1a91c6d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8f8d47ecd646dca1362021e70657b6'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:42.402934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3a862-de18-4b9b-baa6-62ba2e493a16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:42.411367 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3a862-de18-4b9b-baa6-62ba2e493a16 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:42.419796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:42.441833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 854/1702] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:16:41 2022] DELETE /identity/v3/users/5f8f8d47ecd646dca1362021e70657b6 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:42.444878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 849/1703] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:42.445083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b90ab-fb27-4e14-9198-6e1a731b6888 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:42.445588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b90ab-fb27-4e14-9198-6e1a731b6888 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:42.445728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b90ab-fb27-4e14-9198-6e1a731b6888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:42.465990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3a862-de18-4b9b-baa6-62ba2e493a16 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:42.467586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-83e3a862-de18-4b9b-baa6-62ba2e493a16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:42.487520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-ad6b90ab-fb27-4e14-9198-6e1a731b6888 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:42.489265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ad6b90ab-fb27-4e14-9198-6e1a731b6888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:42.632011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 855/1704] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:42.666819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3a862-de18-4b9b-baa6-62ba2e493a16 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:42.674353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b90ab-fb27-4e14-9198-6e1a731b6888 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:42.675853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ad6b90ab-fb27-4e14-9198-6e1a731b6888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:42.741644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:42.828835 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d75f96-39d8-435b-837e-cd2a13f61ec5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:42.829572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75f96-39d8-435b-837e-cd2a13f61ec5 service placement] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:42.830446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75f96-39d8-435b-837e-cd2a13f61ec5 service placement] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:42.830734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75f96-39d8-435b-837e-cd2a13f61ec5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:42.833701 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-48d75f96-39d8-435b-837e-cd2a13f61ec5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:42.880754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d75f96-39d8-435b-837e-cd2a13f61ec5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:42.882395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d75f96-39d8-435b-837e-cd2a13f61ec5 service placement] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:42.903658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 850/1705] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:16:42.910030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b90ab-fb27-4e14-9198-6e1a731b6888 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:42.917324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcd585-7cc6-4836-9bd0-f0855a873232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:42.918004 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcd585-7cc6-4836-9bd0-f0855a873232 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:42.918405 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcd585-7cc6-4836-9bd0-f0855a873232 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:42.918811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcd585-7cc6-4836-9bd0-f0855a873232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:42.933421 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcd585-7cc6-4836-9bd0-f0855a873232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:42.935574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcd585-7cc6-4836-9bd0-f0855a873232 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:42.981541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 856/1706] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:43.251458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f2b5c-eb5a-46fe-8967-56990869d278 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:43.253224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f2b5c-eb5a-46fe-8967-56990869d278 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:43.253686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f2b5c-eb5a-46fe-8967-56990869d278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:43.311184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcd585-7cc6-4836-9bd0-f0855a873232 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:43.312746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5bbcd585-7cc6-4836-9bd0-f0855a873232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:43.392405 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcd585-7cc6-4836-9bd0-f0855a873232 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:43.406854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-3b2f2b5c-eb5a-46fe-8967-56990869d278 None None] MFA Rules not processed for user `b55eed583fac43fb9bfe1366a42bd2fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:43.416130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3b2f2b5c-eb5a-46fe-8967-56990869d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:43.422971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:43.427035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:43.429951 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:43.430281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e admin admin] PATH_INFO: `/v3/projects/15b14c7b75fe4933a275eb9d97c6ce55` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:43.452731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=15b14c7b75fe4933a275eb9d97c6ce55)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:43.463659 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:43.725940 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15b14c7b75fe4933a275eb9d97c6ce55', 'request_id': 'req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:43.748631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15b14c7b75fe4933a275eb9d97c6ce55', 'request_id': 'req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:43.780570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15b14c7b75fe4933a275eb9d97c6ce55', 'request_id': 'req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:43.797742 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 851/1707] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 540 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:43.867414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f2b5c-eb5a-46fe-8967-56990869d278 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:43.869330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3b2f2b5c-eb5a-46fe-8967-56990869d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:43.870375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e admin admin] The token cache is being invalidate because project 15b14c7b75fe4933a275eb9d97c6ce55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:16:43.872931 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:43.878333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e5a272c1-fdec-4674-aa8a-1a62f6d29c5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:43.882335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 857/1708] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:16:43 2022] DELETE /identity/v3/projects/15b14c7b75fe4933a275eb9d97c6ce55 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:16:43.900150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f05773-e100-42e0-96bf-bc284a8697f8 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:43.902554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f05773-e100-42e0-96bf-bc284a8697f8 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:43.902903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f05773-e100-42e0-96bf-bc284a8697f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:43.959661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:44.094253 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-03f05773-e100-42e0-96bf-bc284a8697f8 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:44.095582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-03f05773-e100-42e0-96bf-bc284a8697f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:44.319045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f2b5c-eb5a-46fe-8967-56990869d278 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:44.376452 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 858/1709] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:44.388646 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6766124-bd3e-491b-b9d4-441a98ae6057 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:44.389366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6766124-bd3e-491b-b9d4-441a98ae6057 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:44.389841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6766124-bd3e-491b-b9d4-441a98ae6057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:44.535545 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f6766124-bd3e-491b-b9d4-441a98ae6057 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:44.536965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f6766124-bd3e-491b-b9d4-441a98ae6057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:44.575771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 859/1710] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:44.577889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6766124-bd3e-491b-b9d4-441a98ae6057 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:44.582218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f6766124-bd3e-491b-b9d4-441a98ae6057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:44.611479 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6766124-bd3e-491b-b9d4-441a98ae6057 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:44.628904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:44.630329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:44.630787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:44.631497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] PATH_INFO: `/v3/projects/52f78f78a6054c53bc715e02f4a02f30` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:44.635148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52f78f78a6054c53bc715e02f4a02f30)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:44.640619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:44.704776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfc7ec-6163-4dfc-bef7-adf80b3610bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:44.707559 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfc7ec-6163-4dfc-bef7-adf80b3610bc service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:44.708771 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfc7ec-6163-4dfc-bef7-adf80b3610bc service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:44.709624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfc7ec-6163-4dfc-bef7-adf80b3610bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:44.714640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-dbbfc7ec-6163-4dfc-bef7-adf80b3610bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:44.805482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:44.871418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52f78f78a6054c53bc715e02f4a02f30', 'request_id': 'req-31e35296-5e46-4998-abcb-5720f399cb80'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:44.885120 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52f78f78a6054c53bc715e02f4a02f30', 'request_id': 'req-31e35296-5e46-4998-abcb-5720f399cb80'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:44.907965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52f78f78a6054c53bc715e02f4a02f30', 'request_id': 'req-31e35296-5e46-4998-abcb-5720f399cb80'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:44.930433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] The token cache is being invalidate because project 52f78f78a6054c53bc715e02f4a02f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:16:44.931568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:44.932645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:44.934853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 860/1711] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:16:44 2022] DELETE /identity/v3/projects/52f78f78a6054c53bc715e02f4a02f30 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:44.939031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:44.944311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:44.970678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfc7ec-6163-4dfc-bef7-adf80b3610bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:44.974080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfc7ec-6163-4dfc-bef7-adf80b3610bc service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:45.052477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 852/1712] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 1185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:45.053712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfc7ec-6163-4dfc-bef7-adf80b3610bc service nova] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:45.120407 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfc7ec-6163-4dfc-bef7-adf80b3610bc service nova] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:45.294592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e35296-5e46-4998-abcb-5720f399cb80 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:45.348784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:45.349303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:45.350448 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:45.350764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:45.352506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] PATH_INFO: `/v3/projects/cf2bc6b24c4b46eda718a42805828757` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:45.361722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf2bc6b24c4b46eda718a42805828757)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:45.365438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede26faa-b55f-4bc3-b742-78f9d5472968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:45.366078 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede26faa-b55f-4bc3-b742-78f9d5472968 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:45.366358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede26faa-b55f-4bc3-b742-78f9d5472968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:45.366625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede26faa-b55f-4bc3-b742-78f9d5472968 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:45.367960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede26faa-b55f-4bc3-b742-78f9d5472968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:16:45.367960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede26faa-b55f-4bc3-b742-78f9d5472968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:45.369435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede26faa-b55f-4bc3-b742-78f9d5472968 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:45.373212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:45.409869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 853/1713] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:16:45.409869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede26faa-b55f-4bc3-b742-78f9d5472968 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:45.416527 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ede26faa-b55f-4bc3-b742-78f9d5472968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:45.492468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede26faa-b55f-4bc3-b742-78f9d5472968 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:45.508992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1663c-1bda-4f02-af0b-e4a99e215c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:45.509684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1663c-1bda-4f02-af0b-e4a99e215c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:45.509960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1663c-1bda-4f02-af0b-e4a99e215c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:45.510239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1663c-1bda-4f02-af0b-e4a99e215c9b admin admin] PATH_INFO: `/v3/users/6ef33be1b1c44218b33931e8eb99b577` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:45.513657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1663c-1bda-4f02-af0b-e4a99e215c9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ef33be1b1c44218b33931e8eb99b577)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:45.516730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1663c-1bda-4f02-af0b-e4a99e215c9b admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:45.545147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf2bc6b24c4b46eda718a42805828757', 'request_id': 'req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:45.566345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf2bc6b24c4b46eda718a42805828757', 'request_id': 'req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:45.585592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf2bc6b24c4b46eda718a42805828757', 'request_id': 'req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:45.630127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] The token cache is being invalidate because project cf2bc6b24c4b46eda718a42805828757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:16:45.630451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:45.632050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:45.645041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 861/1714] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:16:44 2022] DELETE /identity/v3/projects/cf2bc6b24c4b46eda718a42805828757 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:45.649328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:45.655011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:45.679087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-06f1663c-1bda-4f02-af0b-e4a99e215c9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ef33be1b1c44218b33931e8eb99b577'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:45.700515 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-06f1663c-1bda-4f02-af0b-e4a99e215c9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ef33be1b1c44218b33931e8eb99b577'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:45.706044 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-06f1663c-1bda-4f02-af0b-e4a99e215c9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ef33be1b1c44218b33931e8eb99b577'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:45.710411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-06f1663c-1bda-4f02-af0b-e4a99e215c9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ef33be1b1c44218b33931e8eb99b577'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:45.751655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-06f1663c-1bda-4f02-af0b-e4a99e215c9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ef33be1b1c44218b33931e8eb99b577'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:45.791958 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 854/1715] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:16:45 2022] DELETE /identity/v3/users/6ef33be1b1c44218b33931e8eb99b577 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:45.793656 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6965f-a06b-452d-8f2c-d3af76b6b4c6 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:45.795439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6965f-a06b-452d-8f2c-d3af76b6b4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:45.799669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6965f-a06b-452d-8f2c-d3af76b6b4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:45.873399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:45.899926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-77c6965f-a06b-452d-8f2c-d3af76b6b4c6 None None] MFA Rules not processed for user `406bdf61f9284ebc8620b21ee414ec91`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:45.901979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-77c6965f-a06b-452d-8f2c-d3af76b6b4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:45.929728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdbe8c-1e5e-4ddc-a9c4-6b8f5de6b2e8 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:46.173914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7267787-46cc-4a92-a92c-a87566d0849d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:46.176054 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7267787-46cc-4a92-a92c-a87566d0849d admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:46.180643 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7267787-46cc-4a92-a92c-a87566d0849d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:46.181671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7267787-46cc-4a92-a92c-a87566d0849d admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:46.186715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7267787-46cc-4a92-a92c-a87566d0849d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:46.201391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7267787-46cc-4a92-a92c-a87566d0849d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:46.211669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 855/1716] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:46.211669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6965f-a06b-452d-8f2c-d3af76b6b4c6 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:46.220599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-77c6965f-a06b-452d-8f2c-d3af76b6b4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:46.379740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6965f-a06b-452d-8f2c-d3af76b6b4c6 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:46.396209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:46.397215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:46.397633 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:46.398051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] PATH_INFO: `/v3/users/7276f5883af54398b3642545110b1adc` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:46.400629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7276f5883af54398b3642545110b1adc)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:46.400907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 862/1717] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:45 2022] POST /identity/v3/projects => generated 355 bytes in 755 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:46.404430 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2b6e4-3198-4329-a9ba-0ca1ebc52b31 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:46.405099 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2b6e4-3198-4329-a9ba-0ca1ebc52b31 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:46.405654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2b6e4-3198-4329-a9ba-0ca1ebc52b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:46.410919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:46.475938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-1bc2b6e4-3198-4329-a9ba-0ca1ebc52b31 None None] MFA Rules not processed for user `406bdf61f9284ebc8620b21ee414ec91`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:46.477696 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1bc2b6e4-3198-4329-a9ba-0ca1ebc52b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:46.550533 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:46.557939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7276f5883af54398b3642545110b1adc'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:46.585866 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7276f5883af54398b3642545110b1adc'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:46.591930 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7276f5883af54398b3642545110b1adc'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:46.594650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7276f5883af54398b3642545110b1adc'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:46.650790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7276f5883af54398b3642545110b1adc'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:46.685926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 863/1718] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:46.685926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2b6e4-3198-4329-a9ba-0ca1ebc52b31 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:46.693580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 856/1719] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:16:46 2022] DELETE /identity/v3/users/7276f5883af54398b3642545110b1adc => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:46.694850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1bc2b6e4-3198-4329-a9ba-0ca1ebc52b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:46.700809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:46.703730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:46.877854 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2b6e4-3198-4329-a9ba-0ca1ebc52b31 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:46.935155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c8776-227f-4668-91f5-f640269dd43d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:46.942550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dc621-a210-4cf3-bb03-f1e91d446ca0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:46.943601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dc621-a210-4cf3-bb03-f1e91d446ca0 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:46.943601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dc621-a210-4cf3-bb03-f1e91d446ca0 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:46.943826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dc621-a210-4cf3-bb03-f1e91d446ca0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:46.945765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1d9dc621-a210-4cf3-bb03-f1e91d446ca0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:46.998348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:47.055537 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04686b9c-c2e5-44dc-a9e0-ae769eef9a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:47.056495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04686b9c-c2e5-44dc-a9e0-ae769eef9a18 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:47.056784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04686b9c-c2e5-44dc-a9e0-ae769eef9a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:47.057060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04686b9c-c2e5-44dc-a9e0-ae769eef9a18 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:47.058065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04686b9c-c2e5-44dc-a9e0-ae769eef9a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:47.062626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04686b9c-c2e5-44dc-a9e0-ae769eef9a18 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:47.084673 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dc621-a210-4cf3-bb03-f1e91d446ca0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:47.086790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dc621-a210-4cf3-bb03-f1e91d446ca0 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:47.115420 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 857/1720] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:46 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:47.135184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 864/1721] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:46 2022] POST /identity/v3/users => generated 278 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:47.138816 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dc621-a210-4cf3-bb03-f1e91d446ca0 service nova] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:47.159527 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dc621-a210-4cf3-bb03-f1e91d446ca0 service nova] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:47.177917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57548a-8ddf-4d99-8a96-3f958011d40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:47.178607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57548a-8ddf-4d99-8a96-3f958011d40c admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:47.178899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57548a-8ddf-4d99-8a96-3f958011d40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:47.179210 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57548a-8ddf-4d99-8a96-3f958011d40c admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:47.179972 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57548a-8ddf-4d99-8a96-3f958011d40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:47.181501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57548a-8ddf-4d99-8a96-3f958011d40c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:47.207146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 858/1722] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:47.213057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04686b9c-c2e5-44dc-a9e0-ae769eef9a18 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:47.274734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04686b9c-c2e5-44dc-a9e0-ae769eef9a18 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:47.295052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795490e6-b80b-4cfc-8312-a4457f26783b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:47.296607 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795490e6-b80b-4cfc-8312-a4457f26783b admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:47.297693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795490e6-b80b-4cfc-8312-a4457f26783b admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:47.298531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795490e6-b80b-4cfc-8312-a4457f26783b admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:47.303939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795490e6-b80b-4cfc-8312-a4457f26783b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:47.308070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795490e6-b80b-4cfc-8312-a4457f26783b admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:47.357772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57548a-8ddf-4d99-8a96-3f958011d40c admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:47.358108 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 865/1723] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:47.369983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57548a-8ddf-4d99-8a96-3f958011d40c admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:47.378113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eca8e-3064-4bdd-a2ab-af39774c83c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:47.378813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eca8e-3064-4bdd-a2ab-af39774c83c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:47.379096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eca8e-3064-4bdd-a2ab-af39774c83c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:47.379460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972eca8e-3064-4bdd-a2ab-af39774c83c8 admin admin] PATH_INFO: `/v3/projects/494c4cf5138f413cb07edab3887ad896/users/65ec656d73d846f489ea4bdd877e058d/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:47.413083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972eca8e-3064-4bdd-a2ab-af39774c83c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=494c4cf5138f413cb07edab3887ad896, user_id=65ec656d73d846f489ea4bdd877e058d, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:47.419300 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972eca8e-3064-4bdd-a2ab-af39774c83c8 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:47.442347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 859/1724] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:47 2022] PUT /identity/v3/projects/494c4cf5138f413cb07edab3887ad896/users/65ec656d73d846f489ea4bdd877e058d/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:16:47.457181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445862c-549d-4915-bfe4-8944ac210131 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:47.464518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445862c-549d-4915-bfe4-8944ac210131 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:47.468795 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445862c-549d-4915-bfe4-8944ac210131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:47.556105 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-e445862c-549d-4915-bfe4-8944ac210131 None None] MFA Rules not processed for user `65ec656d73d846f489ea4bdd877e058d`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:47.557491 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e445862c-549d-4915-bfe4-8944ac210131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:47.755068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eca8e-3064-4bdd-a2ab-af39774c83c8 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:47.757021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-972eca8e-3064-4bdd-a2ab-af39774c83c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:47.823819 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 866/1725] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:47.837150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445862c-549d-4915-bfe4-8944ac210131 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:47.897335 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:48.120690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972eca8e-3064-4bdd-a2ab-af39774c83c8 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:48.140493 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:48.141857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:48.143616 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:48.143910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] PATH_INFO: `/v3/projects/1088a8fb7f2f4544a00955c5ba7d2654` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:48.147038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1088a8fb7f2f4544a00955c5ba7d2654)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:48.154372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:48.213548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445862c-549d-4915-bfe4-8944ac210131 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:48.227870 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05501d5-46b6-451d-a0c8-dfc2fa085dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:48.229290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05501d5-46b6-451d-a0c8-dfc2fa085dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:48.229789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05501d5-46b6-451d-a0c8-dfc2fa085dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:48.230461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05501d5-46b6-451d-a0c8-dfc2fa085dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:48.231943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05501d5-46b6-451d-a0c8-dfc2fa085dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:48.238070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05501d5-46b6-451d-a0c8-dfc2fa085dd9 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:48.305793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 867/1726] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:47 2022] POST /identity/v3/projects => generated 357 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:48.307039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05501d5-46b6-451d-a0c8-dfc2fa085dd9 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:48.310385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b05501d5-46b6-451d-a0c8-dfc2fa085dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:48.368191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1088a8fb7f2f4544a00955c5ba7d2654', 'request_id': 'req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:48.376705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:48.391911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1088a8fb7f2f4544a00955c5ba7d2654', 'request_id': 'req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:48.418031 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1088a8fb7f2f4544a00955c5ba7d2654', 'request_id': 'req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:48.472617 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] The token cache is being invalidate because project 1088a8fb7f2f4544a00955c5ba7d2654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:16:48.472919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:48.475469 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:48.498749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 860/1727] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:16:47 2022] DELETE /identity/v3/projects/1088a8fb7f2f4544a00955c5ba7d2654 => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:48.498749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:48.509750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:48.511432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05501d5-46b6-451d-a0c8-dfc2fa085dd9 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:48.525378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f256f65-0112-43f7-9689-e54539176a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:48.526846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f256f65-0112-43f7-9689-e54539176a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:48.530446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f256f65-0112-43f7-9689-e54539176a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:48.531644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f256f65-0112-43f7-9689-e54539176a86 admin admin] PATH_INFO: `/v3/projects/c8df7cad7caf4de5ac2b27970319ce94` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:48.536383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f256f65-0112-43f7-9689-e54539176a86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8df7cad7caf4de5ac2b27970319ce94)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:48.541974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f256f65-0112-43f7-9689-e54539176a86 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:48.679344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1f256f65-0112-43f7-9689-e54539176a86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8df7cad7caf4de5ac2b27970319ce94', 'request_id': 'req-1f256f65-0112-43f7-9689-e54539176a86'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:48.713202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1f256f65-0112-43f7-9689-e54539176a86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8df7cad7caf4de5ac2b27970319ce94', 'request_id': 'req-1f256f65-0112-43f7-9689-e54539176a86'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:48.740487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ee3fc-dbb7-47a7-8070-d0e17b7fb950 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:48.742771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1f256f65-0112-43f7-9689-e54539176a86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8df7cad7caf4de5ac2b27970319ce94', 'request_id': 'req-1f256f65-0112-43f7-9689-e54539176a86'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:48.769469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1f256f65-0112-43f7-9689-e54539176a86 admin admin] The token cache is being invalidate because project c8df7cad7caf4de5ac2b27970319ce94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:16:48.770076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1f256f65-0112-43f7-9689-e54539176a86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:48.771646 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1f256f65-0112-43f7-9689-e54539176a86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:16:48.781577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 868/1728] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:16:48 2022] DELETE /identity/v3/projects/c8df7cad7caf4de5ac2b27970319ce94 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:48.787085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363c165-b219-4ee4-8c1d-3c18a229fdca None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:48.787772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363c165-b219-4ee4-8c1d-3c18a229fdca None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:48.788247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363c165-b219-4ee4-8c1d-3c18a229fdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:48.849817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-7363c165-b219-4ee4-8c1d-3c18a229fdca None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:48.855903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7363c165-b219-4ee4-8c1d-3c18a229fdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:48.905143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b408df-de24-44ad-b4b0-e6ea614ec6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:48.909060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b408df-de24-44ad-b4b0-e6ea614ec6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:48.909536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b408df-de24-44ad-b4b0-e6ea614ec6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:48.910223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b408df-de24-44ad-b4b0-e6ea614ec6c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:48.911231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b408df-de24-44ad-b4b0-e6ea614ec6c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:48.914831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b408df-de24-44ad-b4b0-e6ea614ec6c9 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:48.927841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:48.986422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 861/1729] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:48 2022] POST /identity/v3/users => generated 285 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:48.987866 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb534e-7752-476c-8314-602c30aa56b2 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:48.988155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb534e-7752-476c-8314-602c30aa56b2 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:48.988431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb534e-7752-476c-8314-602c30aa56b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:49.055256 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-41fb534e-7752-476c-8314-602c30aa56b2 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:49.060796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-41fb534e-7752-476c-8314-602c30aa56b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:49.092039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 862/1730] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:49.095191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb534e-7752-476c-8314-602c30aa56b2 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:49.104315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-41fb534e-7752-476c-8314-602c30aa56b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:49.141796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb534e-7752-476c-8314-602c30aa56b2 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:49.157854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be934f96-ff71-439a-b832-9090c5190830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:49.158499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be934f96-ff71-439a-b832-9090c5190830 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:49.158779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be934f96-ff71-439a-b832-9090c5190830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:49.159052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be934f96-ff71-439a-b832-9090c5190830 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:49.159817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be934f96-ff71-439a-b832-9090c5190830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:49.163384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be934f96-ff71-439a-b832-9090c5190830 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:49.193419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 863/1731] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:49.195149 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45f2d6-55f7-4de4-9796-fb75c303e11d None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:49.195507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45f2d6-55f7-4de4-9796-fb75c303e11d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:49.195723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45f2d6-55f7-4de4-9796-fb75c303e11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:49.208856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 869/1732] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:49.219852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12270456-db40-43f3-8911-bcb5ace3066f None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:49.220049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12270456-db40-43f3-8911-bcb5ace3066f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:49.220263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12270456-db40-43f3-8911-bcb5ace3066f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:49.254946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-0c45f2d6-55f7-4de4-9796-fb75c303e11d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:49.256456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0c45f2d6-55f7-4de4-9796-fb75c303e11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:49.290569 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 864/1733] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:49.291521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45f2d6-55f7-4de4-9796-fb75c303e11d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:49.316378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45f2d6-55f7-4de4-9796-fb75c303e11d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:49.349082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f71628-1929-4227-8a5e-4f0814ac93c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:49.349810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f71628-1929-4227-8a5e-4f0814ac93c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:49.350099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f71628-1929-4227-8a5e-4f0814ac93c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:49.350383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f71628-1929-4227-8a5e-4f0814ac93c4 admin admin] PATH_INFO: `/v3/projects/2d1b30f4841e456f822629d36233c075/users/4941088d4eea469ea5b0ead03f340b2b/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:49.369536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-12270456-db40-43f3-8911-bcb5ace3066f None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:49.377245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-12270456-db40-43f3-8911-bcb5ace3066f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:49.417333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f71628-1929-4227-8a5e-4f0814ac93c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d1b30f4841e456f822629d36233c075, user_id=4941088d4eea469ea5b0ead03f340b2b, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:49.423774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f71628-1929-4227-8a5e-4f0814ac93c4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:49.463061 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 865/1734] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:49 2022] PUT /identity/v3/projects/2d1b30f4841e456f822629d36233c075/users/4941088d4eea469ea5b0ead03f340b2b/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:49.466338 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:49.466754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34183d3-3f6b-43d1-98d9-fa916a29c0b8 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:49.467555 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34183d3-3f6b-43d1-98d9-fa916a29c0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:49.467936 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34183d3-3f6b-43d1-98d9-fa916a29c0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:49.536529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f34183d3-3f6b-43d1-98d9-fa916a29c0b8 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:49.542403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f34183d3-3f6b-43d1-98d9-fa916a29c0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:49.627483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 870/1735] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:16:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:49.630084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12270456-db40-43f3-8911-bcb5ace3066f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:49.682158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12270456-db40-43f3-8911-bcb5ace3066f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:49.812983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 866/1736] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:49.815895 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34183d3-3f6b-43d1-98d9-fa916a29c0b8 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:49.839059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34183d3-3f6b-43d1-98d9-fa916a29c0b8 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:49.853568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd25085-341a-4c4e-acd5-9a5380ec9237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:49.853568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd25085-341a-4c4e-acd5-9a5380ec9237 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:49.856497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd25085-341a-4c4e-acd5-9a5380ec9237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:49.856934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd25085-341a-4c4e-acd5-9a5380ec9237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:49.858090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd25085-341a-4c4e-acd5-9a5380ec9237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:49.862352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd25085-341a-4c4e-acd5-9a5380ec9237 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:49.881909 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad821f6-d3bf-48fe-ba16-ebf543c516ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:49.883984 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad821f6-d3bf-48fe-ba16-ebf543c516ab admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:49.884771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad821f6-d3bf-48fe-ba16-ebf543c516ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:49.887495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad821f6-d3bf-48fe-ba16-ebf543c516ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:49.889289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad821f6-d3bf-48fe-ba16-ebf543c516ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:16:49.891625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad821f6-d3bf-48fe-ba16-ebf543c516ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:49.900001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad821f6-d3bf-48fe-ba16-ebf543c516ab admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:49.906855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 867/1737] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:49.909520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd25085-341a-4c4e-acd5-9a5380ec9237 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:49.965293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd25085-341a-4c4e-acd5-9a5380ec9237 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:49.968977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 871/1738] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:16:49.969846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad821f6-d3bf-48fe-ba16-ebf543c516ab admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:49.976527 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20f1f2-3ace-4a3d-923d-957ccc173273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:49.978921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-8ad821f6-d3bf-48fe-ba16-ebf543c516ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:49.979037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20f1f2-3ace-4a3d-923d-957ccc173273 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:49.979393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20f1f2-3ace-4a3d-923d-957ccc173273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:49.979667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20f1f2-3ace-4a3d-923d-957ccc173273 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:49.980546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20f1f2-3ace-4a3d-923d-957ccc173273 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:16:49.981508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20f1f2-3ace-4a3d-923d-957ccc173273 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:49.984683 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20f1f2-3ace-4a3d-923d-957ccc173273 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:50.012147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 868/1739] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:16:50.014633 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20f1f2-3ace-4a3d-923d-957ccc173273 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:50.031224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20f1f2-3ace-4a3d-923d-957ccc173273 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:50.038450 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe7c1e-cc64-4b08-b868-3b6ecae4426f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:50.039838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe7c1e-cc64-4b08-b868-3b6ecae4426f admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:50.040261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe7c1e-cc64-4b08-b868-3b6ecae4426f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:50.040545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe7c1e-cc64-4b08-b868-3b6ecae4426f admin admin] PATH_INFO: `/v3/projects/2d1b30f4841e456f822629d36233c075/users/4941088d4eea469ea5b0ead03f340b2b/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:50.042756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe7c1e-cc64-4b08-b868-3b6ecae4426f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d1b30f4841e456f822629d36233c075, user_id=4941088d4eea469ea5b0ead03f340b2b, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:50.047132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe7c1e-cc64-4b08-b868-3b6ecae4426f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:50.078189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 869/1740] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:50 2022] PUT /identity/v3/projects/2d1b30f4841e456f822629d36233c075/users/4941088d4eea469ea5b0ead03f340b2b/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:50.079482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c252c-4655-45df-83d4-186fbdfdb5db None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:50.079778 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c252c-4655-45df-83d4-186fbdfdb5db None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:50.080061 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c252c-4655-45df-83d4-186fbdfdb5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:50.124079 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-353c252c-4655-45df-83d4-186fbdfdb5db None None] MFA Rules not processed for user `406bdf61f9284ebc8620b21ee414ec91`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:50.126332 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-353c252c-4655-45df-83d4-186fbdfdb5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:50.245003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad821f6-d3bf-48fe-ba16-ebf543c516ab admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:50.393695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:50.404225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 870/1741] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:50.405822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c252c-4655-45df-83d4-186fbdfdb5db None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:50.444151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49470f7e-804a-4378-b840-bad28cab7b2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:50.445467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49470f7e-804a-4378-b840-bad28cab7b2d service swift] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:50.446022 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49470f7e-804a-4378-b840-bad28cab7b2d service swift] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:50.446495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49470f7e-804a-4378-b840-bad28cab7b2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:50.482846 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:50.548865 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49470f7e-804a-4378-b840-bad28cab7b2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:50.550812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49470f7e-804a-4378-b840-bad28cab7b2d service swift] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:50.563375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 872/1742] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:16:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:16:50.564622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49470f7e-804a-4378-b840-bad28cab7b2d service swift] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:50.836613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49470f7e-804a-4378-b840-bad28cab7b2d service swift] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:50.880460 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da39de-fb8c-48ff-a251-62171d456356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:50.881104 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da39de-fb8c-48ff-a251-62171d456356 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:50.881391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da39de-fb8c-48ff-a251-62171d456356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:50.881668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da39de-fb8c-48ff-a251-62171d456356 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:50.882318 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da39de-fb8c-48ff-a251-62171d456356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:50.902732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da39de-fb8c-48ff-a251-62171d456356 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:50.957240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c252c-4655-45df-83d4-186fbdfdb5db None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:51.004521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208b3bb-2142-412f-8f90-935b5d4a05b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:51.005577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208b3bb-2142-412f-8f90-935b5d4a05b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:51.006061 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208b3bb-2142-412f-8f90-935b5d4a05b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:51.006623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208b3bb-2142-412f-8f90-935b5d4a05b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:51.007855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208b3bb-2142-412f-8f90-935b5d4a05b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:51.039879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208b3bb-2142-412f-8f90-935b5d4a05b8 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:51.054298 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 873/1743] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:50 2022] POST /identity/v3/projects => generated 385 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:51.055731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877be61b-48dc-471f-bdaa-a74534ce4832 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:51.056021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877be61b-48dc-471f-bdaa-a74534ce4832 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:51.056305 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877be61b-48dc-471f-bdaa-a74534ce4832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:51.204246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-877be61b-48dc-471f-bdaa-a74534ce4832 None None] MFA Rules not processed for user `4941088d4eea469ea5b0ead03f340b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:51.211371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-877be61b-48dc-471f-bdaa-a74534ce4832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:51.269853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 871/1744] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:50 2022] POST /identity/v3/projects => generated 381 bytes in 865 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:51.271469 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208b3bb-2142-412f-8f90-935b5d4a05b8 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:51.539240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:51.554134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 874/1745] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:51 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:51.559943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877be61b-48dc-471f-bdaa-a74534ce4832 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:51.686968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208b3bb-2142-412f-8f90-935b5d4a05b8 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:51.716073 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41305f8f-fdb4-4f05-ae1b-f38b2f2ef606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:51.717377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41305f8f-fdb4-4f05-ae1b-f38b2f2ef606 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:51.721136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41305f8f-fdb4-4f05-ae1b-f38b2f2ef606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:51.725020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41305f8f-fdb4-4f05-ae1b-f38b2f2ef606 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:51.726846 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41305f8f-fdb4-4f05-ae1b-f38b2f2ef606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:51.747834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41305f8f-fdb4-4f05-ae1b-f38b2f2ef606 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:51.762966 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877be61b-48dc-471f-bdaa-a74534ce4832 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:51.787816 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bdae2-d464-429f-be5a-e51484d95766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:51.788705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bdae2-d464-429f-be5a-e51484d95766 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:51.791160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bdae2-d464-429f-be5a-e51484d95766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:51.791712 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bdae2-d464-429f-be5a-e51484d95766 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:51.792529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bdae2-d464-429f-be5a-e51484d95766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:51.797503 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bdae2-d464-429f-be5a-e51484d95766 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:51.868313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 875/1746] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:51 2022] POST /identity/v3/users => generated 291 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:51.875764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bdae2-d464-429f-be5a-e51484d95766 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:51.884626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-669bdae2-d464-429f-be5a-e51484d95766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:51.920501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 872/1747] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:51 2022] POST /identity/v3/users => generated 293 bytes in 650 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:51.926314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41305f8f-fdb4-4f05-ae1b-f38b2f2ef606 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:52.046913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41305f8f-fdb4-4f05-ae1b-f38b2f2ef606 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:52.086709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78707d0e-1f09-4056-b29c-0335bbfa96b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:52.088273 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78707d0e-1f09-4056-b29c-0335bbfa96b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:52.088594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78707d0e-1f09-4056-b29c-0335bbfa96b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:52.088872 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78707d0e-1f09-4056-b29c-0335bbfa96b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:52.093384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78707d0e-1f09-4056-b29c-0335bbfa96b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:52.096719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78707d0e-1f09-4056-b29c-0335bbfa96b1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:52.124046 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 873/1748] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:52.125344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78707d0e-1f09-4056-b29c-0335bbfa96b1 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:52.162704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78707d0e-1f09-4056-b29c-0335bbfa96b1 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:52.181093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0570e-cb63-4161-a55e-c60024ae88c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:52.182132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0570e-cb63-4161-a55e-c60024ae88c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:52.182570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0570e-cb63-4161-a55e-c60024ae88c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:52.183222 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0570e-cb63-4161-a55e-c60024ae88c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:52.184120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0570e-cb63-4161-a55e-c60024ae88c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:52.184400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bdae2-d464-429f-be5a-e51484d95766 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:52.189198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0570e-cb63-4161-a55e-c60024ae88c1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:52.195093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63db074-bfdd-47b1-a1d6-5e44428a4ea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:52.196580 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63db074-bfdd-47b1-a1d6-5e44428a4ea5 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:52.197536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63db074-bfdd-47b1-a1d6-5e44428a4ea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:52.198143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63db074-bfdd-47b1-a1d6-5e44428a4ea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:52.233980 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 874/1749] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:52.237392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0570e-cb63-4161-a55e-c60024ae88c1 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:52.267252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0570e-cb63-4161-a55e-c60024ae88c1 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:52.281859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397ba1f-c922-4bb3-bbb5-90ef132ebae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:52.282861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397ba1f-c922-4bb3-bbb5-90ef132ebae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:52.283764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397ba1f-c922-4bb3-bbb5-90ef132ebae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:52.284466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397ba1f-c922-4bb3-bbb5-90ef132ebae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:52.287449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397ba1f-c922-4bb3-bbb5-90ef132ebae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:52.290861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397ba1f-c922-4bb3-bbb5-90ef132ebae7 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:52.324480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 875/1750] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:52.325714 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397ba1f-c922-4bb3-bbb5-90ef132ebae7 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:52.351197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397ba1f-c922-4bb3-bbb5-90ef132ebae7 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:52.369604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16e584-2e83-499b-a2da-1cad7b0b1662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:52.370445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16e584-2e83-499b-a2da-1cad7b0b1662 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:52.370903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16e584-2e83-499b-a2da-1cad7b0b1662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:52.371894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16e584-2e83-499b-a2da-1cad7b0b1662 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:52.374258 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16e584-2e83-499b-a2da-1cad7b0b1662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:52.379640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16e584-2e83-499b-a2da-1cad7b0b1662 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:52.400241 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63db074-bfdd-47b1-a1d6-5e44428a4ea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:52.402127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63db074-bfdd-47b1-a1d6-5e44428a4ea5 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:52.412971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 876/1751] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:52.414528 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16e584-2e83-499b-a2da-1cad7b0b1662 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:52.442076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 876/1752] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:51 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:52.444286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63db074-bfdd-47b1-a1d6-5e44428a4ea5 service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:52.457691 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16e584-2e83-499b-a2da-1cad7b0b1662 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:52.477280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a8fb3-8ce3-40c0-a4ce-49ae37b1e878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:52.478221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a8fb3-8ce3-40c0-a4ce-49ae37b1e878 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:52.478591 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a8fb3-8ce3-40c0-a4ce-49ae37b1e878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:52.478779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a8fb3-8ce3-40c0-a4ce-49ae37b1e878 admin admin] PATH_INFO: `/v3/projects/9c6bac4e2c0546ce8a1848eacc375ce6/users/eac53832ac5a4bfab495af280dc74cd6/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:52.480951 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63db074-bfdd-47b1-a1d6-5e44428a4ea5 service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:52.501799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7ba14-e803-4fdb-b101-1dc7f02c1607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:52.505517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7ba14-e803-4fdb-b101-1dc7f02c1607 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:52.505914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7ba14-e803-4fdb-b101-1dc7f02c1607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:52.506369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7ba14-e803-4fdb-b101-1dc7f02c1607 admin admin] PATH_INFO: `/v3/projects/a0a9ce6cc6f142f0ae27e1a440c5ec9f/users/8089759b0704409e99fc76b40137a653/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:52.546955 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a8fb3-8ce3-40c0-a4ce-49ae37b1e878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c6bac4e2c0546ce8a1848eacc375ce6, user_id=eac53832ac5a4bfab495af280dc74cd6, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:52.550877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a8fb3-8ce3-40c0-a4ce-49ae37b1e878 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:52.605097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 877/1753] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:52 2022] PUT /identity/v3/projects/9c6bac4e2c0546ce8a1848eacc375ce6/users/eac53832ac5a4bfab495af280dc74cd6/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:52.615379 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b47ced-49ae-407d-946e-a5cfa031c33c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:52.615749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b47ced-49ae-407d-946e-a5cfa031c33c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:52.616032 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b47ced-49ae-407d-946e-a5cfa031c33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:52.626265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7ba14-e803-4fdb-b101-1dc7f02c1607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a9ce6cc6f142f0ae27e1a440c5ec9f, user_id=8089759b0704409e99fc76b40137a653, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:52.633563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7ba14-e803-4fdb-b101-1dc7f02c1607 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:52.661528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 877/1754] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:52 2022] PUT /identity/v3/projects/a0a9ce6cc6f142f0ae27e1a440c5ec9f/users/8089759b0704409e99fc76b40137a653/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:52.662844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7ba14-e803-4fdb-b101-1dc7f02c1607 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:52.674654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3ae7ba14-e803-4fdb-b101-1dc7f02c1607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:52.696233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:52.700189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-a1b47ced-49ae-407d-946e-a5cfa031c33c None None] MFA Rules not processed for user `eac53832ac5a4bfab495af280dc74cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:52.701536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a1b47ced-49ae-407d-946e-a5cfa031c33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:52.709814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:52.801762 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7ba14-e803-4fdb-b101-1dc7f02c1607 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:52.811861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3fba4-a530-449c-82a8-8c6112613f2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:52.812510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3fba4-a530-449c-82a8-8c6112613f2e service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:52.812952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3fba4-a530-449c-82a8-8c6112613f2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:52.813239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3fba4-a530-449c-82a8-8c6112613f2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:52.816553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-84c3fba4-a530-449c-82a8-8c6112613f2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:52.868635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 878/1755] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:52.870299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d550bb4-0114-44f6-a0ab-dd7998141833 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:52.870588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d550bb4-0114-44f6-a0ab-dd7998141833 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:52.872520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d550bb4-0114-44f6-a0ab-dd7998141833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:52.953715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-6d550bb4-0114-44f6-a0ab-dd7998141833 None None] MFA Rules not processed for user `8089759b0704409e99fc76b40137a653`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:52.954589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6d550bb4-0114-44f6-a0ab-dd7998141833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:53.068748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3fba4-a530-449c-82a8-8c6112613f2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:53.077694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3fba4-a530-449c-82a8-8c6112613f2e service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:53.147269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 878/1756] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:53.148423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3fba4-a530-449c-82a8-8c6112613f2e service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:53.330802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 879/1757] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:53.332137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d550bb4-0114-44f6-a0ab-dd7998141833 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:53.425778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3fba4-a530-449c-82a8-8c6112613f2e service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:53.452350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0e50f-480b-42a6-bd55-59e5bdb8f76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:53.457695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0e50f-480b-42a6-bd55-59e5bdb8f76a admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:53.458504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0e50f-480b-42a6-bd55-59e5bdb8f76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:53.459531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0e50f-480b-42a6-bd55-59e5bdb8f76a admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:53.460985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0e50f-480b-42a6-bd55-59e5bdb8f76a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:53.471053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0e50f-480b-42a6-bd55-59e5bdb8f76a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:53.652335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 879/1758] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:53 2022] POST /identity/v3/projects => generated 381 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:53.658942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0e50f-480b-42a6-bd55-59e5bdb8f76a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:53.749751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d550bb4-0114-44f6-a0ab-dd7998141833 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:53.938171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:53.971129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:53.986217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf99f6-7463-4da6-962e-7dedd0eae7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:53.987123 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf99f6-7463-4da6-962e-7dedd0eae7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:53.992970 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf99f6-7463-4da6-962e-7dedd0eae7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:53.993536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf99f6-7463-4da6-962e-7dedd0eae7d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:53.994486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf99f6-7463-4da6-962e-7dedd0eae7d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:54.019969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf99f6-7463-4da6-962e-7dedd0eae7d0 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:54.023879 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0e50f-480b-42a6-bd55-59e5bdb8f76a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:54.052817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7a32-f091-4160-9ff2-9ec77dc14626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:54.055521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7a32-f091-4160-9ff2-9ec77dc14626 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:54.056041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7a32-f091-4160-9ff2-9ec77dc14626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:54.057619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7a32-f091-4160-9ff2-9ec77dc14626 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:54.058547 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b7a32-f091-4160-9ff2-9ec77dc14626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:54.070499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b7a32-f091-4160-9ff2-9ec77dc14626 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:54.122937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 880/1759] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:53 2022] POST /identity/v3/users => generated 297 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:54.126218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7a32-f091-4160-9ff2-9ec77dc14626 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:54.145453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 880/1760] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:53 2022] POST /identity/v3/projects => generated 385 bytes in 811 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:54.180029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf99f6-7463-4da6-962e-7dedd0eae7d0 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:54.193087 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7a32-f091-4160-9ff2-9ec77dc14626 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:54.376528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed04c85-2b50-4ba6-83ea-32941d2e0fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:54.377181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed04c85-2b50-4ba6-83ea-32941d2e0fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:54.377466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed04c85-2b50-4ba6-83ea-32941d2e0fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:54.379257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed04c85-2b50-4ba6-83ea-32941d2e0fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:54.379989 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed04c85-2b50-4ba6-83ea-32941d2e0fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:54.381698 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed04c85-2b50-4ba6-83ea-32941d2e0fe7 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:54.418972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 881/1761] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:16:54.425638 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed04c85-2b50-4ba6-83ea-32941d2e0fe7 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:54.462991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed04c85-2b50-4ba6-83ea-32941d2e0fe7 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:54.470798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44258200-f48a-4169-b9e2-257fa9890176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:54.473685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44258200-f48a-4169-b9e2-257fa9890176 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:54.473685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44258200-f48a-4169-b9e2-257fa9890176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:54.473685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44258200-f48a-4169-b9e2-257fa9890176 admin admin] PATH_INFO: `/v3/projects/c1e778b332f3428babbabce28e672723/users/4a85d4654d1f4d78848745af2ea5c005/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:54.518442 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf99f6-7463-4da6-962e-7dedd0eae7d0 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:54.536625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e6f27-50c0-4124-b53b-20db2e8b28e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:54.537829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e6f27-50c0-4124-b53b-20db2e8b28e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:54.538805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e6f27-50c0-4124-b53b-20db2e8b28e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:54.539445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e6f27-50c0-4124-b53b-20db2e8b28e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:54.540967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e6f27-50c0-4124-b53b-20db2e8b28e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:54.550397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e6f27-50c0-4124-b53b-20db2e8b28e2 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:54.551334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44258200-f48a-4169-b9e2-257fa9890176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e778b332f3428babbabce28e672723, user_id=4a85d4654d1f4d78848745af2ea5c005, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:54.563535 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44258200-f48a-4169-b9e2-257fa9890176 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:54.642580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 881/1762] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:54 2022] POST /identity/v3/users => generated 293 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:54.642795 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 882/1763] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:54 2022] PUT /identity/v3/projects/c1e778b332f3428babbabce28e672723/users/4a85d4654d1f4d78848745af2ea5c005/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:54.642883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e6f27-50c0-4124-b53b-20db2e8b28e2 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:54.644453 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44258200-f48a-4169-b9e2-257fa9890176 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:54.677923 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e6f27-50c0-4124-b53b-20db2e8b28e2 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:54.984274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44258200-f48a-4169-b9e2-257fa9890176 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:54.994478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d0109-683c-416b-9856-dd670d7073e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:54.995345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d0109-683c-416b-9856-dd670d7073e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:54.995678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d0109-683c-416b-9856-dd670d7073e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:54.996168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d0109-683c-416b-9856-dd670d7073e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:54.996924 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d0109-683c-416b-9856-dd670d7073e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:55.001805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d0109-683c-416b-9856-dd670d7073e0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:55.039513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 883/1764] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:55.042286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b45269-efa1-489a-9296-d518d136a264 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:55.042481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b45269-efa1-489a-9296-d518d136a264 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:55.042809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b45269-efa1-489a-9296-d518d136a264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:55.088782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922751e-0393-403a-aac8-9c8754a4b841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:55.092957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922751e-0393-403a-aac8-9c8754a4b841 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:55.096544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922751e-0393-403a-aac8-9c8754a4b841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:55.097002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922751e-0393-403a-aac8-9c8754a4b841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:55.101108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922751e-0393-403a-aac8-9c8754a4b841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:55.105730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922751e-0393-403a-aac8-9c8754a4b841 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:55.124660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 882/1765] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:55.125859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922751e-0393-403a-aac8-9c8754a4b841 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:55.141938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922751e-0393-403a-aac8-9c8754a4b841 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:55.149898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-e7b45269-efa1-489a-9296-d518d136a264 None None] MFA Rules not processed for user `de05c08e30dd4fff93f1c97aac9553d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:55.151092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e7b45269-efa1-489a-9296-d518d136a264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:55.157356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d7146-91f7-4358-a32c-305ba13bd436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:55.158271 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d7146-91f7-4358-a32c-305ba13bd436 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:55.158650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d7146-91f7-4358-a32c-305ba13bd436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:55.159045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d7146-91f7-4358-a32c-305ba13bd436 admin admin] PATH_INFO: `/v3/projects/c1e778b332f3428babbabce28e672723/users/4a85d4654d1f4d78848745af2ea5c005/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:55.163932 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d7146-91f7-4358-a32c-305ba13bd436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e778b332f3428babbabce28e672723, user_id=4a85d4654d1f4d78848745af2ea5c005, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:55.169682 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d7146-91f7-4358-a32c-305ba13bd436 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:55.216434 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 883/1766] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:55 2022] PUT /identity/v3/projects/c1e778b332f3428babbabce28e672723/users/4a85d4654d1f4d78848745af2ea5c005/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:55.217502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d7146-91f7-4358-a32c-305ba13bd436 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:55.350475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:55.386638 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 884/1767] 10.222.0.72 () {58 vars in 1015 bytes} [Wed Sep 7 13:16:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:55.395585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0674c-d529-430c-b4a2-42d8ef2d723e None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:55.396125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0674c-d529-430c-b4a2-42d8ef2d723e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:55.416617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0674c-d529-430c-b4a2-42d8ef2d723e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:55.470167 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:55.563185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d7146-91f7-4358-a32c-305ba13bd436 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:55.587952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-6ba0674c-d529-430c-b4a2-42d8ef2d723e None None] MFA Rules not processed for user `4a85d4654d1f4d78848745af2ea5c005`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:55.608131 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-6ba0674c-d529-430c-b4a2-42d8ef2d723e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:55.609950 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f64ca-c8cb-464f-a8ae-cc96a162aa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:55.613109 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f64ca-c8cb-464f-a8ae-cc96a162aa22 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:55.613381 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f64ca-c8cb-464f-a8ae-cc96a162aa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:55.613652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f64ca-c8cb-464f-a8ae-cc96a162aa22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:55.616692 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f64ca-c8cb-464f-a8ae-cc96a162aa22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:55.622368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f64ca-c8cb-464f-a8ae-cc96a162aa22 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:55.650281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 884/1768] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:55.652482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f64ca-c8cb-464f-a8ae-cc96a162aa22 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:55.656534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d59f64ca-c8cb-464f-a8ae-cc96a162aa22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:55.818158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 885/1769] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:55.819159 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0674c-d529-430c-b4a2-42d8ef2d723e None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:55.839004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f64ca-c8cb-464f-a8ae-cc96a162aa22 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:55.849019 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48879e1d-fe46-4cbb-bd63-e2fde4f45223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:55.850871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48879e1d-fe46-4cbb-bd63-e2fde4f45223 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:55.852346 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48879e1d-fe46-4cbb-bd63-e2fde4f45223 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:55.853634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48879e1d-fe46-4cbb-bd63-e2fde4f45223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:55.856820 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0674c-d529-430c-b4a2-42d8ef2d723e None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:55.871847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6daa8c-0bfd-41f3-94b9-757861c8358c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:55.872951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48879e1d-fe46-4cbb-bd63-e2fde4f45223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:55.875532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6daa8c-0bfd-41f3-94b9-757861c8358c admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:55.876055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6daa8c-0bfd-41f3-94b9-757861c8358c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:55.876535 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48879e1d-fe46-4cbb-bd63-e2fde4f45223 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:55.877061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6daa8c-0bfd-41f3-94b9-757861c8358c admin admin] PATH_INFO: `/v3/projects/28b6dc71255f42d989c859c8881876fc/users/f17af21d5cd64544952c617bbad0fa47/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:55.909334 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 885/1770] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:55.918795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48879e1d-fe46-4cbb-bd63-e2fde4f45223 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:55.944506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6daa8c-0bfd-41f3-94b9-757861c8358c admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b6dc71255f42d989c859c8881876fc, user_id=f17af21d5cd64544952c617bbad0fa47, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:55.968695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6daa8c-0bfd-41f3-94b9-757861c8358c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:56.000541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 886/1771] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:55 2022] PUT /identity/v3/projects/28b6dc71255f42d989c859c8881876fc/users/f17af21d5cd64544952c617bbad0fa47/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:56.018200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae78a9-29fe-462e-9179-900a3d769fbc None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:56.018738 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae78a9-29fe-462e-9179-900a3d769fbc None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:56.019389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae78a9-29fe-462e-9179-900a3d769fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:56.096528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-35ae78a9-29fe-462e-9179-900a3d769fbc None None] MFA Rules not processed for user `f17af21d5cd64544952c617bbad0fa47`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:56.098630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-35ae78a9-29fe-462e-9179-900a3d769fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:56.299536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 887/1772] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:16:56.300985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e0657-ef32-4ce7-b389-388f00a5ea24 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:56.303817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e0657-ef32-4ce7-b389-388f00a5ea24 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:56.304270 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e0657-ef32-4ce7-b389-388f00a5ea24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:56.354245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48879e1d-fe46-4cbb-bd63-e2fde4f45223 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:56.363788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-842e0657-ef32-4ce7-b389-388f00a5ea24 None None] MFA Rules not processed for user `b55eed583fac43fb9bfe1366a42bd2fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:56.365418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-842e0657-ef32-4ce7-b389-388f00a5ea24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:56.494021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:56.541823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 888/1773] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:16:56.544167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e0657-ef32-4ce7-b389-388f00a5ea24 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:56.682465 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8487b2-2158-4620-b7c2-37dec8d5f3c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:56.683556 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8487b2-2158-4620-b7c2-37dec8d5f3c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:56.686474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8487b2-2158-4620-b7c2-37dec8d5f3c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:56.687035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8487b2-2158-4620-b7c2-37dec8d5f3c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:56.795878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e0657-ef32-4ce7-b389-388f00a5ea24 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:56.811615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b717e-7dc8-4f15-9c31-adbd537bccb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:56.815256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b717e-7dc8-4f15-9c31-adbd537bccb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:56.815485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b717e-7dc8-4f15-9c31-adbd537bccb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:56.818596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b717e-7dc8-4f15-9c31-adbd537bccb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:56.818596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b717e-7dc8-4f15-9c31-adbd537bccb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:56.825150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b717e-7dc8-4f15-9c31-adbd537bccb7 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:56.920016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:56.964085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8487b2-2158-4620-b7c2-37dec8d5f3c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:56.966773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8487b2-2158-4620-b7c2-37dec8d5f3c0 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:56.993780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 886/1774] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 1083 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:57.003629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8487b2-2158-4620-b7c2-37dec8d5f3c0 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:57.009211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 889/1775] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:16:56 2022] POST /identity/v3/projects => generated 385 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:57.034663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8487b2-2158-4620-b7c2-37dec8d5f3c0 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:57.358423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b84210-ba1d-41e3-ad3e-6b3cb6997e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:57.359384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b84210-ba1d-41e3-ad3e-6b3cb6997e00 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:57.359384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b84210-ba1d-41e3-ad3e-6b3cb6997e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:57.360150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b84210-ba1d-41e3-ad3e-6b3cb6997e00 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:57.360572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b84210-ba1d-41e3-ad3e-6b3cb6997e00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:57.362662 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b84210-ba1d-41e3-ad3e-6b3cb6997e00 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:57.543836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b717e-7dc8-4f15-9c31-adbd537bccb7 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:57.547290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 887/1776] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:16:57 2022] POST /identity/v3/users => generated 299 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:16:57.591205 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b717e-7dc8-4f15-9c31-adbd537bccb7 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:57.642137 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d596d-8433-4521-8ee8-131e2ce891f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:57.643243 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d596d-8433-4521-8ee8-131e2ce891f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:57.643663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d596d-8433-4521-8ee8-131e2ce891f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:57.644150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d596d-8433-4521-8ee8-131e2ce891f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:57.645331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d596d-8433-4521-8ee8-131e2ce891f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:57.665328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d596d-8433-4521-8ee8-131e2ce891f5 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:57.698495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 890/1777] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:57.705636 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b84210-ba1d-41e3-ad3e-6b3cb6997e00 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:57.751819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b84210-ba1d-41e3-ad3e-6b3cb6997e00 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:57.781230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbf78d-7c13-46bc-bbcf-d340b6cd33f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:57.782057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbf78d-7c13-46bc-bbcf-d340b6cd33f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:57.782360 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbf78d-7c13-46bc-bbcf-d340b6cd33f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:57.782620 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbf78d-7c13-46bc-bbcf-d340b6cd33f3 admin admin] PATH_INFO: `/v3/projects/04ebd3301dbe4be18e75eadc223b2bd3/users/5a88c69ffed74dc2b6ce68f6f04c59e7/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:57.856068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbf78d-7c13-46bc-bbcf-d340b6cd33f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ebd3301dbe4be18e75eadc223b2bd3, user_id=5a88c69ffed74dc2b6ce68f6f04c59e7, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:57.877685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbf78d-7c13-46bc-bbcf-d340b6cd33f3 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:57.932520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 888/1778] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:57 2022] PUT /identity/v3/projects/04ebd3301dbe4be18e75eadc223b2bd3/users/5a88c69ffed74dc2b6ce68f6f04c59e7/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:57.933927 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d596d-8433-4521-8ee8-131e2ce891f5 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:57.959151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d596d-8433-4521-8ee8-131e2ce891f5 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:58.056738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbf78d-7c13-46bc-bbcf-d340b6cd33f3 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:58.232921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d1d87-1146-4108-9f77-0fafee497777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:58.233663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d1d87-1146-4108-9f77-0fafee497777 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:58.233959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d1d87-1146-4108-9f77-0fafee497777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:58.234226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d1d87-1146-4108-9f77-0fafee497777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:58.236836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbf78d-7c13-46bc-bbcf-d340b6cd33f3 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:58.237793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d1d87-1146-4108-9f77-0fafee497777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:58.239531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d1d87-1146-4108-9f77-0fafee497777 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:58.249021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53739a3-717e-4c0b-b4f2-944e5fd7e6b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:58.249651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53739a3-717e-4c0b-b4f2-944e5fd7e6b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:58.250004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53739a3-717e-4c0b-b4f2-944e5fd7e6b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:58.250224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53739a3-717e-4c0b-b4f2-944e5fd7e6b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:58.296649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 891/1779] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:16:58.302422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d1d87-1146-4108-9f77-0fafee497777 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:58.341998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d1d87-1146-4108-9f77-0fafee497777 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:58.370260 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d999623-39ab-4cae-a069-88af084e1937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:58.371493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d999623-39ab-4cae-a069-88af084e1937 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:58.372793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d999623-39ab-4cae-a069-88af084e1937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:58.374734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d999623-39ab-4cae-a069-88af084e1937 admin admin] PATH_INFO: `/v3/projects/04ebd3301dbe4be18e75eadc223b2bd3/users/5a88c69ffed74dc2b6ce68f6f04c59e7/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:58.382311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d999623-39ab-4cae-a069-88af084e1937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ebd3301dbe4be18e75eadc223b2bd3, user_id=5a88c69ffed74dc2b6ce68f6f04c59e7, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:58.388365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d999623-39ab-4cae-a069-88af084e1937 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:58.454922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 892/1780] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:16:58 2022] PUT /identity/v3/projects/04ebd3301dbe4be18e75eadc223b2bd3/users/5a88c69ffed74dc2b6ce68f6f04c59e7/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:16:58.466200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ec972-c037-4ca2-b802-748fc51f40c7 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:58.466740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ec972-c037-4ca2-b802-748fc51f40c7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:58.467151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ec972-c037-4ca2-b802-748fc51f40c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:58.513486 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-8f5ec972-c037-4ca2-b802-748fc51f40c7 None None] MFA Rules not processed for user `5a88c69ffed74dc2b6ce68f6f04c59e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:16:58.515971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-8f5ec972-c037-4ca2-b802-748fc51f40c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:16:58.633369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53739a3-717e-4c0b-b4f2-944e5fd7e6b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:58.637729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53739a3-717e-4c0b-b4f2-944e5fd7e6b9 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:58.686977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 889/1781] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:16:58.696047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:58.795074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 893/1782] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:16:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:16:58.863237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53739a3-717e-4c0b-b4f2-944e5fd7e6b9 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:16:58.878713 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53739a3-717e-4c0b-b4f2-944e5fd7e6b9 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:16:58.890613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:16:59.046710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc96a0-8587-44ee-90ba-da9e31d32d8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:16:59.048120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc96a0-8587-44ee-90ba-da9e31d32d8b service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:16:59.049378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc96a0-8587-44ee-90ba-da9e31d32d8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:16:59.050025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc96a0-8587-44ee-90ba-da9e31d32d8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:16:59.073166 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc96a0-8587-44ee-90ba-da9e31d32d8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:16:59.083356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc96a0-8587-44ee-90ba-da9e31d32d8b service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:16:59.106905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 890/1783] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:16:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:00.607911 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ec972-c037-4ca2-b802-748fc51f40c7 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:00.621625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ec972-c037-4ca2-b802-748fc51f40c7 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:00.637573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5fea8-988d-4842-b8bf-4feefbbb5c67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:00.638243 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5fea8-988d-4842-b8bf-4feefbbb5c67 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:00.638519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5fea8-988d-4842-b8bf-4feefbbb5c67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:00.638788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5fea8-988d-4842-b8bf-4feefbbb5c67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:00.752164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5fea8-988d-4842-b8bf-4feefbbb5c67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:00.753816 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5fea8-988d-4842-b8bf-4feefbbb5c67 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:00.775602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 894/1784] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:00.841956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b17a-af74-4c79-b311-d25005ca9b04 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:00.843681 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b17a-af74-4c79-b311-d25005ca9b04 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:00.844250 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b17a-af74-4c79-b311-d25005ca9b04 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:00.846797 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 891/1785] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:17:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:17:00.863057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36460b51-2030-428e-a02f-e05bbe632792 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:00.863351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36460b51-2030-428e-a02f-e05bbe632792 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:00.863845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36460b51-2030-428e-a02f-e05bbe632792 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:00.865418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 895/1786] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:17:00.889758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d07ba2-5e76-4aae-91f9-30d334cf312c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:00.890106 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d07ba2-5e76-4aae-91f9-30d334cf312c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:00.890327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d07ba2-5e76-4aae-91f9-30d334cf312c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:00.952209 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d99e3-baa6-45cf-9275-6627380ba10a None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:00.952742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d99e3-baa6-45cf-9275-6627380ba10a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:00.953214 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d99e3-baa6-45cf-9275-6627380ba10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:01.013723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-05d07ba2-5e76-4aae-91f9-30d334cf312c None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:01.015075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-05d07ba2-5e76-4aae-91f9-30d334cf312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:01.020186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a28d99e3-baa6-45cf-9275-6627380ba10a None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:01.026404 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a28d99e3-baa6-45cf-9275-6627380ba10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:01.161202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 892/1787] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:17:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:01.175814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d07ba2-5e76-4aae-91f9-30d334cf312c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:01.177337 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-05d07ba2-5e76-4aae-91f9-30d334cf312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:01.192977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d07ba2-5e76-4aae-91f9-30d334cf312c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:01.205874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0611b5-da8c-4973-a4bc-733fb07afe8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:01.205874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0611b5-da8c-4973-a4bc-733fb07afe8c service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:01.205874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0611b5-da8c-4973-a4bc-733fb07afe8c service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:01.205874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0611b5-da8c-4973-a4bc-733fb07afe8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:01.217859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0611b5-da8c-4973-a4bc-733fb07afe8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:01.220036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0611b5-da8c-4973-a4bc-733fb07afe8c service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:01.225208 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 896/1788] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:17:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:01.248106 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 893/1789] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:17:01.368773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d99e3-baa6-45cf-9275-6627380ba10a None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:01.385352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d99e3-baa6-45cf-9275-6627380ba10a None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:01.395686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f5e4b-dadd-4b5e-b6b6-6dec7767eac1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:01.396465 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f5e4b-dadd-4b5e-b6b6-6dec7767eac1 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:01.397389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f5e4b-dadd-4b5e-b6b6-6dec7767eac1 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:01.397389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f5e4b-dadd-4b5e-b6b6-6dec7767eac1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:01.409741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f5e4b-dadd-4b5e-b6b6-6dec7767eac1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:01.411955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f5e4b-dadd-4b5e-b6b6-6dec7767eac1 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:01.430472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 897/1790] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:09.929321 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0611b5-da8c-4973-a4bc-733fb07afe8c service nova] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:09.995708 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0611b5-da8c-4973-a4bc-733fb07afe8c service nova] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:10.000699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:10.001351 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:10.001631 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:10.001909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] PATH_INFO: `/v3/users/eac53832ac5a4bfab495af280dc74cd6` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:10.003062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac53832ac5a4bfab495af280dc74cd6)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:10.005082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:10.045845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:10.062012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac53832ac5a4bfab495af280dc74cd6'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:10.070903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac53832ac5a4bfab495af280dc74cd6'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:10.071528 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac53832ac5a4bfab495af280dc74cd6'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:10.072187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac53832ac5a4bfab495af280dc74cd6'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:10.093447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac53832ac5a4bfab495af280dc74cd6'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:10.131452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 894/1791] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:09 2022] DELETE /identity/v3/users/eac53832ac5a4bfab495af280dc74cd6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:10.133615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3812f-3c13-42b7-8331-188a360ba788 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:10.133615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3812f-3c13-42b7-8331-188a360ba788 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:10.144571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3812f-3c13-42b7-8331-188a360ba788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:10.158239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-31e3812f-3c13-42b7-8331-188a360ba788 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:10.160225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-31e3812f-3c13-42b7-8331-188a360ba788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:10.247475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:10.280900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 898/1792] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:10.424124 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:10.426949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:10.650879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd943cd-502d-4638-82a5-04a3542384b0 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:10.663855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fcede-e194-43d2-8495-02eac2ce520d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:10.665791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fcede-e194-43d2-8495-02eac2ce520d service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:10.666080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fcede-e194-43d2-8495-02eac2ce520d service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:10.666743 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fcede-e194-43d2-8495-02eac2ce520d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:10.696100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fcede-e194-43d2-8495-02eac2ce520d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:10.699911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fcede-e194-43d2-8495-02eac2ce520d service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:10.726323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 895/1793] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:12.421649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3812f-3c13-42b7-8331-188a360ba788 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:12.429197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-31e3812f-3c13-42b7-8331-188a360ba788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:12.457142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3812f-3c13-42b7-8331-188a360ba788 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:12.469826 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:12.470662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:12.471981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:12.473580 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] PATH_INFO: `/v3/projects/9c6bac4e2c0546ce8a1848eacc375ce6` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:12.475819 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c6bac4e2c0546ce8a1848eacc375ce6)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:12.478547 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:12.677615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c6bac4e2c0546ce8a1848eacc375ce6', 'request_id': 'req-afa85ef1-5b34-4804-a663-860316bbfaa3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:12.694687 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c6bac4e2c0546ce8a1848eacc375ce6', 'request_id': 'req-afa85ef1-5b34-4804-a663-860316bbfaa3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:12.711310 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c6bac4e2c0546ce8a1848eacc375ce6', 'request_id': 'req-afa85ef1-5b34-4804-a663-860316bbfaa3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:12.737567 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] The token cache is being invalidate because project 9c6bac4e2c0546ce8a1848eacc375ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:12.738156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:12.739008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:12.741661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 899/1794] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:12 2022] DELETE /identity/v3/projects/9c6bac4e2c0546ce8a1848eacc375ce6 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:12.745951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fcede-e194-43d2-8495-02eac2ce520d service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:12.769355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:12.773792 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:12.777549 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fcede-e194-43d2-8495-02eac2ce520d service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:12.947097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa85ef1-5b34-4804-a663-860316bbfaa3 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:12.954686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432e756-b36d-4d9e-a531-9c9f307ba07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:12.955786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432e756-b36d-4d9e-a531-9c9f307ba07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:12.956556 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432e756-b36d-4d9e-a531-9c9f307ba07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:12.956995 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432e756-b36d-4d9e-a531-9c9f307ba07c admin admin] PATH_INFO: `/v3/users/4a85d4654d1f4d78848745af2ea5c005` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:12.960170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432e756-b36d-4d9e-a531-9c9f307ba07c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a85d4654d1f4d78848745af2ea5c005)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:12.962746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432e756-b36d-4d9e-a531-9c9f307ba07c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:12.962896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b381b5c-bdf3-43d8-ba4d-a54241931d9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:12.963542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b381b5c-bdf3-43d8-ba4d-a54241931d9b service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:12.964330 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b381b5c-bdf3-43d8-ba4d-a54241931d9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:12.964610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b381b5c-bdf3-43d8-ba4d-a54241931d9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:12.966519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3b381b5c-bdf3-43d8-ba4d-a54241931d9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:13.122750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b381b5c-bdf3-43d8-ba4d-a54241931d9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:13.131401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b381b5c-bdf3-43d8-ba4d-a54241931d9b service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:13.132519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6432e756-b36d-4d9e-a531-9c9f307ba07c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a85d4654d1f4d78848745af2ea5c005'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:13.143530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:13.152903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6432e756-b36d-4d9e-a531-9c9f307ba07c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a85d4654d1f4d78848745af2ea5c005'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:13.155319 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6432e756-b36d-4d9e-a531-9c9f307ba07c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a85d4654d1f4d78848745af2ea5c005'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:13.156233 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6432e756-b36d-4d9e-a531-9c9f307ba07c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a85d4654d1f4d78848745af2ea5c005'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:13.177351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 900/1795] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:13.178881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6432e756-b36d-4d9e-a531-9c9f307ba07c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a85d4654d1f4d78848745af2ea5c005'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:13.201626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 896/1796] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:12 2022] DELETE /identity/v3/users/4a85d4654d1f4d78848745af2ea5c005 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:15.737439 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b381b5c-bdf3-43d8-ba4d-a54241931d9b service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:15.751708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3b381b5c-bdf3-43d8-ba4d-a54241931d9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:15.781701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b381b5c-bdf3-43d8-ba4d-a54241931d9b service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:15.874485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:15.953020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:15.953996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:15.954365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:15.954764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a admin admin] PATH_INFO: `/v3/projects/c1e778b332f3428babbabce28e672723` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:15.956656 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1e778b332f3428babbabce28e672723)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:15.961215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:16.048921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e778b332f3428babbabce28e672723', 'request_id': 'req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:16.061037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e778b332f3428babbabce28e672723', 'request_id': 'req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:16.079648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e778b332f3428babbabce28e672723', 'request_id': 'req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:16.107014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a admin admin] The token cache is being invalidate because project c1e778b332f3428babbabce28e672723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:16.108085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:16.109839 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f6bead2e-e77d-43c0-ba10-9fe7e19a171a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:16.112362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 901/1797] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:15 2022] DELETE /identity/v3/projects/c1e778b332f3428babbabce28e672723 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:16.117378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feebf27-50bb-4968-87ba-6a6d7a26cf9f None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:16.118512 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feebf27-50bb-4968-87ba-6a6d7a26cf9f None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:16.118512 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feebf27-50bb-4968-87ba-6a6d7a26cf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:16.159365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-9feebf27-50bb-4968-87ba-6a6d7a26cf9f None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:16.160661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-9feebf27-50bb-4968-87ba-6a6d7a26cf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:16.310781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 897/1798] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:16.320592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc7add-5f90-4823-9150-9acbeefdddfe None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:16.321730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc7add-5f90-4823-9150-9acbeefdddfe None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:16.322748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc7add-5f90-4823-9150-9acbeefdddfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:16.376134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-61bc7add-5f90-4823-9150-9acbeefdddfe None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:16.377506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-61bc7add-5f90-4823-9150-9acbeefdddfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:16.390704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 902/1799] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:16.395041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feebf27-50bb-4968-87ba-6a6d7a26cf9f None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:16.413449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feebf27-50bb-4968-87ba-6a6d7a26cf9f None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:16.429829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4b3fc-be11-4077-afb4-b07aca6a322a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:16.430438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4b3fc-be11-4077-afb4-b07aca6a322a admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:16.431014 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4b3fc-be11-4077-afb4-b07aca6a322a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:16.432782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4b3fc-be11-4077-afb4-b07aca6a322a admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:16.433747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4b3fc-be11-4077-afb4-b07aca6a322a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:17:16.443795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4b3fc-be11-4077-afb4-b07aca6a322a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:16.445743 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4b3fc-be11-4077-afb4-b07aca6a322a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:16.479891 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc7add-5f90-4823-9150-9acbeefdddfe None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:16.486861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 898/1800] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:17:16.494993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc7add-5f90-4823-9150-9acbeefdddfe None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:16.503068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c30776-4f2b-4a4c-93e5-172d4d8221b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:16.504130 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30776-4f2b-4a4c-93e5-172d4d8221b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:16.504417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30776-4f2b-4a4c-93e5-172d4d8221b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:16.504774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30776-4f2b-4a4c-93e5-172d4d8221b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:16.505534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c30776-4f2b-4a4c-93e5-172d4d8221b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:16.507908 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c30776-4f2b-4a4c-93e5-172d4d8221b7 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:16.567284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 903/1801] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:16 2022] POST /identity/v3/projects => generated 409 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:17:16.568819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4b3fc-be11-4077-afb4-b07aca6a322a admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:16.584764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4b3fc-be11-4077-afb4-b07aca6a322a admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:16.681254 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c30776-4f2b-4a4c-93e5-172d4d8221b7 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:16.683327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d0c30776-4f2b-4a4c-93e5-172d4d8221b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:16.697055 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a5974-c776-4345-b3fc-ece6351d56a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:16.698144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a5974-c776-4345-b3fc-ece6351d56a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:16.698433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a5974-c776-4345-b3fc-ece6351d56a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:16.699082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a5974-c776-4345-b3fc-ece6351d56a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:16.700180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a5974-c776-4345-b3fc-ece6351d56a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:16.702649 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a5974-c776-4345-b3fc-ece6351d56a2 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:16.736098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 899/1802] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:16 2022] POST /identity/v3/users => generated 305 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:16.740342 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a5974-c776-4345-b3fc-ece6351d56a2 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:16.759715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a5974-c776-4345-b3fc-ece6351d56a2 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:16.770800 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80014b16-c64c-42e8-b67a-17b834c3fc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:16.771746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80014b16-c64c-42e8-b67a-17b834c3fc55 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:16.772227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80014b16-c64c-42e8-b67a-17b834c3fc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:16.772517 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80014b16-c64c-42e8-b67a-17b834c3fc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:16.773376 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80014b16-c64c-42e8-b67a-17b834c3fc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:16.774667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c30776-4f2b-4a4c-93e5-172d4d8221b7 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:16.775482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80014b16-c64c-42e8-b67a-17b834c3fc55 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:16.780710 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071486c5-fa64-4585-b511-55e0c3df7b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:16.781767 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071486c5-fa64-4585-b511-55e0c3df7b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:16.782328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071486c5-fa64-4585-b511-55e0c3df7b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:16.782820 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071486c5-fa64-4585-b511-55e0c3df7b4c admin admin] PATH_INFO: `/v3/users/8089759b0704409e99fc76b40137a653` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:16.784916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071486c5-fa64-4585-b511-55e0c3df7b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8089759b0704409e99fc76b40137a653)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:16.787275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071486c5-fa64-4585-b511-55e0c3df7b4c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:16.792195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 900/1803] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:17:16.797404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80014b16-c64c-42e8-b67a-17b834c3fc55 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:16.808999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80014b16-c64c-42e8-b67a-17b834c3fc55 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:16.814213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce3c3c-a81b-433c-95e1-c17993d7e488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:16.814725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce3c3c-a81b-433c-95e1-c17993d7e488 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:16.815000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce3c3c-a81b-433c-95e1-c17993d7e488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:16.815340 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce3c3c-a81b-433c-95e1-c17993d7e488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:16.815949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce3c3c-a81b-433c-95e1-c17993d7e488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:16.817703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce3c3c-a81b-433c-95e1-c17993d7e488 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:16.829794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 901/1804] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:16.840365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce3c3c-a81b-433c-95e1-c17993d7e488 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:16.853584 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce3c3c-a81b-433c-95e1-c17993d7e488 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:16.859182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdffbbd-4f2c-4488-8677-7643335cbb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:16.860348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdffbbd-4f2c-4488-8677-7643335cbb6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:16.860685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdffbbd-4f2c-4488-8677-7643335cbb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:16.861646 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdffbbd-4f2c-4488-8677-7643335cbb6c admin admin] PATH_INFO: `/v3/projects/e4fe2d2e29b54244b9233b50de580d59/users/c8fde53a9f56451ca82d3b5b3450619c/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:16.862677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:16.879635 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-071486c5-fa64-4585-b511-55e0c3df7b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8089759b0704409e99fc76b40137a653'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:16.882390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdffbbd-4f2c-4488-8677-7643335cbb6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fe2d2e29b54244b9233b50de580d59, user_id=c8fde53a9f56451ca82d3b5b3450619c, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:16.886210 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdffbbd-4f2c-4488-8677-7643335cbb6c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:16.914900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-071486c5-fa64-4585-b511-55e0c3df7b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8089759b0704409e99fc76b40137a653'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:16.919914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:16.931592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-071486c5-fa64-4585-b511-55e0c3df7b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8089759b0704409e99fc76b40137a653'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:16.933405 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-071486c5-fa64-4585-b511-55e0c3df7b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8089759b0704409e99fc76b40137a653'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:16.934976 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-071486c5-fa64-4585-b511-55e0c3df7b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8089759b0704409e99fc76b40137a653'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:16.943009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 902/1805] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:16 2022] PUT /identity/v3/projects/e4fe2d2e29b54244b9233b50de580d59/users/c8fde53a9f56451ca82d3b5b3450619c/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:16.954338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa1c80-4160-4160-8592-cb5c018da808 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:16.954580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa1c80-4160-4160-8592-cb5c018da808 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:16.954943 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa1c80-4160-4160-8592-cb5c018da808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:16.965448 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 904/1806] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:16 2022] DELETE /identity/v3/users/8089759b0704409e99fc76b40137a653 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:16.974325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb086-4159-4b4d-b7a8-7002fce904a8 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:16.974612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb086-4159-4b4d-b7a8-7002fce904a8 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:16.974890 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb086-4159-4b4d-b7a8-7002fce904a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:16.995084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ddaa1c80-4160-4160-8592-cb5c018da808 None None] MFA Rules not processed for user `c8fde53a9f56451ca82d3b5b3450619c`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:16.997013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ddaa1c80-4160-4160-8592-cb5c018da808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:17.007684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-eaecb086-4159-4b4d-b7a8-7002fce904a8 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:17.011353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-eaecb086-4159-4b4d-b7a8-7002fce904a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:17.091379 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 903/1807] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:16 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:17.095003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa1c80-4160-4160-8592-cb5c018da808 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:17.097136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ddaa1c80-4160-4160-8592-cb5c018da808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:17.114259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 905/1808] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:17.126451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecb086-4159-4b4d-b7a8-7002fce904a8 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:17.128224 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-eaecb086-4159-4b4d-b7a8-7002fce904a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:17.215549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecb086-4159-4b4d-b7a8-7002fce904a8 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:17.221006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b05ab-e7f3-4660-9d2c-97d2418eb4c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:17.222730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b05ab-e7f3-4660-9d2c-97d2418eb4c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:17.223036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b05ab-e7f3-4660-9d2c-97d2418eb4c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:17.223342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b05ab-e7f3-4660-9d2c-97d2418eb4c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:17.226176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa1c80-4160-4160-8592-cb5c018da808 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:17.232980 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503d948-ae31-4a06-87d3-77cc61e17732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:17.233687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503d948-ae31-4a06-87d3-77cc61e17732 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:17.234069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503d948-ae31-4a06-87d3-77cc61e17732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:17.234668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503d948-ae31-4a06-87d3-77cc61e17732 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:17.235703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503d948-ae31-4a06-87d3-77cc61e17732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:17.237517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b05ab-e7f3-4660-9d2c-97d2418eb4c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:17.238233 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503d948-ae31-4a06-87d3-77cc61e17732 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:17.239309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b05ab-e7f3-4660-9d2c-97d2418eb4c1 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:17.276479 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 906/1809] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:17:17.292102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 904/1810] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:17 2022] POST /identity/v3/projects => generated 407 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:17.296337 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b05ab-e7f3-4660-9d2c-97d2418eb4c1 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:17.312563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b05ab-e7f3-4660-9d2c-97d2418eb4c1 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:17.401867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:17.415451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb565f-3aa4-4301-81c7-249fedcfa836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:17.417152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb565f-3aa4-4301-81c7-249fedcfa836 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:17.417892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb565f-3aa4-4301-81c7-249fedcfa836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:17.418929 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb565f-3aa4-4301-81c7-249fedcfa836 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:17.421181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb565f-3aa4-4301-81c7-249fedcfa836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:17.424865 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb565f-3aa4-4301-81c7-249fedcfa836 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:17.456237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 907/1811] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:17 2022] POST /identity/v3/users => generated 310 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:17.457576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503d948-ae31-4a06-87d3-77cc61e17732 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:17.469861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503d948-ae31-4a06-87d3-77cc61e17732 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:17.476583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834886b-08cf-4b5a-bb47-77faca060e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:17.477957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834886b-08cf-4b5a-bb47-77faca060e78 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:17.478412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834886b-08cf-4b5a-bb47-77faca060e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:17.478731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834886b-08cf-4b5a-bb47-77faca060e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:17.479727 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834886b-08cf-4b5a-bb47-77faca060e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:17.482643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834886b-08cf-4b5a-bb47-77faca060e78 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:17.495125 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 905/1812] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:17:17.499670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb565f-3aa4-4301-81c7-249fedcfa836 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:17.510952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb565f-3aa4-4301-81c7-249fedcfa836 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:17.516871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c084baf-84b6-467d-97e9-196f94783eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:17.518277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c084baf-84b6-467d-97e9-196f94783eae admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:17.518277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c084baf-84b6-467d-97e9-196f94783eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:17.518277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c084baf-84b6-467d-97e9-196f94783eae admin admin] PATH_INFO: `/v3/projects/befeae50edc24147b1f79ec23333c483/users/c225d229a13f4c2395c359dc23b9a1c2/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:17.539570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c084baf-84b6-467d-97e9-196f94783eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=befeae50edc24147b1f79ec23333c483, user_id=c225d229a13f4c2395c359dc23b9a1c2, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:17.543811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c084baf-84b6-467d-97e9-196f94783eae admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:17.565854 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 908/1813] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:17 2022] PUT /identity/v3/projects/befeae50edc24147b1f79ec23333c483/users/c225d229a13f4c2395c359dc23b9a1c2/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:17.567575 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834886b-08cf-4b5a-bb47-77faca060e78 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:17.584257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834886b-08cf-4b5a-bb47-77faca060e78 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:17.722965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b291db-38ba-4222-ac28-f765a5fcb7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:17.723579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b291db-38ba-4222-ac28-f765a5fcb7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:17.723862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b291db-38ba-4222-ac28-f765a5fcb7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:17.724205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b291db-38ba-4222-ac28-f765a5fcb7f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:17.725561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b291db-38ba-4222-ac28-f765a5fcb7f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:17.731368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b291db-38ba-4222-ac28-f765a5fcb7f4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:17.736774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:17.752585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 906/1814] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:17.757682 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c084baf-84b6-467d-97e9-196f94783eae admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:17.770185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c084baf-84b6-467d-97e9-196f94783eae admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:17.775535 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f7b32-1071-4f2c-98a5-4e1f9821a888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:17.776982 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f7b32-1071-4f2c-98a5-4e1f9821a888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:17.777117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f7b32-1071-4f2c-98a5-4e1f9821a888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:17.777344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f7b32-1071-4f2c-98a5-4e1f9821a888 admin admin] PATH_INFO: `/v3/projects/befeae50edc24147b1f79ec23333c483/users/c225d229a13f4c2395c359dc23b9a1c2/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:17.779461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f7b32-1071-4f2c-98a5-4e1f9821a888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=befeae50edc24147b1f79ec23333c483, user_id=c225d229a13f4c2395c359dc23b9a1c2, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:17.782521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f7b32-1071-4f2c-98a5-4e1f9821a888 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:17.799156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 909/1815] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:17 2022] PUT /identity/v3/projects/befeae50edc24147b1f79ec23333c483/users/c225d229a13f4c2395c359dc23b9a1c2/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:17.814665 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f720a43-2fa7-460e-882e-99738f04d148 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:17.815215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f720a43-2fa7-460e-882e-99738f04d148 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:17.815729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f720a43-2fa7-460e-882e-99738f04d148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:17.867131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8f720a43-2fa7-460e-882e-99738f04d148 None None] MFA Rules not processed for user `c225d229a13f4c2395c359dc23b9a1c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:17.868425 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8f720a43-2fa7-460e-882e-99738f04d148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:17.946423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f7b32-1071-4f2c-98a5-4e1f9821a888 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:17.949786 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-8f0f7b32-1071-4f2c-98a5-4e1f9821a888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:17.983878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 907/1816] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:17 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:18.007594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f720a43-2fa7-460e-882e-99738f04d148 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:18.009393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8f720a43-2fa7-460e-882e-99738f04d148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:18.081531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f7b32-1071-4f2c-98a5-4e1f9821a888 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:18.090927 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:18.092375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:18.093684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:18.095432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] PATH_INFO: `/v3/projects/a0a9ce6cc6f142f0ae27e1a440c5ec9f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:18.097160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a9ce6cc6f142f0ae27e1a440c5ec9f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:18.100510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f720a43-2fa7-460e-882e-99738f04d148 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:18.100683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:18.110560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe316b54-6e39-4aaa-9e54-31e274a2cae0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:18.111584 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe316b54-6e39-4aaa-9e54-31e274a2cae0 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:18.112007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe316b54-6e39-4aaa-9e54-31e274a2cae0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:18.112286 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe316b54-6e39-4aaa-9e54-31e274a2cae0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:18.123129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe316b54-6e39-4aaa-9e54-31e274a2cae0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:18.124625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe316b54-6e39-4aaa-9e54-31e274a2cae0 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:18.137403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 908/1817] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:18 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:18.175654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a9ce6cc6f142f0ae27e1a440c5ec9f', 'request_id': 'req-8a5c5b8f-5979-4731-9d38-c9049233428e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:18.187879 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a9ce6cc6f142f0ae27e1a440c5ec9f', 'request_id': 'req-8a5c5b8f-5979-4731-9d38-c9049233428e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:18.200191 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a9ce6cc6f142f0ae27e1a440c5ec9f', 'request_id': 'req-8a5c5b8f-5979-4731-9d38-c9049233428e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:18.221526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] The token cache is being invalidate because project a0a9ce6cc6f142f0ae27e1a440c5ec9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:18.222294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:18.224236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:18.226511 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 910/1818] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:17 2022] DELETE /identity/v3/projects/a0a9ce6cc6f142f0ae27e1a440c5ec9f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:18.231007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe316b54-6e39-4aaa-9e54-31e274a2cae0 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:18.251419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe316b54-6e39-4aaa-9e54-31e274a2cae0 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:18.316404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:18.317070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:18.317349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:18.317622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] PATH_INFO: `/v3/users/f17af21d5cd64544952c617bbad0fa47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:18.318799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f17af21d5cd64544952c617bbad0fa47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:18.320783 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:18.363053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:18.394578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f17af21d5cd64544952c617bbad0fa47'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:18.400127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:18.402518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:18.406816 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f17af21d5cd64544952c617bbad0fa47'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:18.408129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f17af21d5cd64544952c617bbad0fa47'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:18.409934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f17af21d5cd64544952c617bbad0fa47'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:18.429275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f17af21d5cd64544952c617bbad0fa47'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:18.446751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 909/1819] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:18 2022] DELETE /identity/v3/users/f17af21d5cd64544952c617bbad0fa47 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:18.485529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:18.511962 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c5b8f-5979-4731-9d38-c9049233428e admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:18.597942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a4556-1335-4187-aa7e-aa8c1d398d1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:18.598341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a4556-1335-4187-aa7e-aa8c1d398d1e service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:18.598521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a4556-1335-4187-aa7e-aa8c1d398d1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:18.598794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a4556-1335-4187-aa7e-aa8c1d398d1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:18.600750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1b5a4556-1335-4187-aa7e-aa8c1d398d1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:18.694001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a4556-1335-4187-aa7e-aa8c1d398d1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:18.697382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a4556-1335-4187-aa7e-aa8c1d398d1e service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:18.713449 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 911/1820] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:18 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:17:18.906481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:18.908174 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:19.034658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538cc4d-a260-44b3-a820-f512e51cd655 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:19.040874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:19.042067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:19.042634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:19.043254 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] PATH_INFO: `/v3/projects/28b6dc71255f42d989c859c8881876fc` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:19.044909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] RBAC: Authorizing `identity:delete_project(project_id=28b6dc71255f42d989c859c8881876fc)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:19.051909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:19.146220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28b6dc71255f42d989c859c8881876fc', 'request_id': 'req-3220b4b4-04e9-49c9-a13d-df7326d5c27c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:19.176650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28b6dc71255f42d989c859c8881876fc', 'request_id': 'req-3220b4b4-04e9-49c9-a13d-df7326d5c27c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:19.184225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28b6dc71255f42d989c859c8881876fc', 'request_id': 'req-3220b4b4-04e9-49c9-a13d-df7326d5c27c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:19.212352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] The token cache is being invalidate because project 28b6dc71255f42d989c859c8881876fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:19.213155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:19.219722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:19.227867 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 910/1821] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:18 2022] DELETE /identity/v3/projects/28b6dc71255f42d989c859c8881876fc => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:19.233576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a4556-1335-4187-aa7e-aa8c1d398d1e service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:19.254798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a4556-1335-4187-aa7e-aa8c1d398d1e service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:19.337543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e46f0-7993-4b76-a6f7-034f62b7d256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:19.338798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e46f0-7993-4b76-a6f7-034f62b7d256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:19.339085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e46f0-7993-4b76-a6f7-034f62b7d256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:19.339396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e46f0-7993-4b76-a6f7-034f62b7d256 admin admin] PATH_INFO: `/v3/users/5a88c69ffed74dc2b6ce68f6f04c59e7` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:19.340630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e46f0-7993-4b76-a6f7-034f62b7d256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a88c69ffed74dc2b6ce68f6f04c59e7)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:19.342624 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e46f0-7993-4b76-a6f7-034f62b7d256 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:19.406786 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-884e46f0-7993-4b76-a6f7-034f62b7d256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a88c69ffed74dc2b6ce68f6f04c59e7'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:19.414871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:19.431830 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-884e46f0-7993-4b76-a6f7-034f62b7d256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a88c69ffed74dc2b6ce68f6f04c59e7'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:19.442937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-884e46f0-7993-4b76-a6f7-034f62b7d256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a88c69ffed74dc2b6ce68f6f04c59e7'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:19.443848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-884e46f0-7993-4b76-a6f7-034f62b7d256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a88c69ffed74dc2b6ce68f6f04c59e7'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:19.444628 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-884e46f0-7993-4b76-a6f7-034f62b7d256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a88c69ffed74dc2b6ce68f6f04c59e7'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:19.466347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 912/1822] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:19 2022] DELETE /identity/v3/users/5a88c69ffed74dc2b6ce68f6f04c59e7 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:19.813850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:19.815461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:19.831453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220b4b4-04e9-49c9-a13d-df7326d5c27c admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:19.883899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:19.920503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83329ba0-915e-4ee2-940a-daf4e4591b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:19.921523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83329ba0-915e-4ee2-940a-daf4e4591b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:19.921907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83329ba0-915e-4ee2-940a-daf4e4591b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:19.922188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83329ba0-915e-4ee2-940a-daf4e4591b5f admin admin] PATH_INFO: `/v3/projects/04ebd3301dbe4be18e75eadc223b2bd3` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:19.923575 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83329ba0-915e-4ee2-940a-daf4e4591b5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ebd3301dbe4be18e75eadc223b2bd3)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:19.926147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83329ba0-915e-4ee2-940a-daf4e4591b5f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:19.999484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-83329ba0-915e-4ee2-940a-daf4e4591b5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ebd3301dbe4be18e75eadc223b2bd3', 'request_id': 'req-83329ba0-915e-4ee2-940a-daf4e4591b5f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:20.010521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-83329ba0-915e-4ee2-940a-daf4e4591b5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ebd3301dbe4be18e75eadc223b2bd3', 'request_id': 'req-83329ba0-915e-4ee2-940a-daf4e4591b5f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:20.021278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-83329ba0-915e-4ee2-940a-daf4e4591b5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ebd3301dbe4be18e75eadc223b2bd3', 'request_id': 'req-83329ba0-915e-4ee2-940a-daf4e4591b5f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:20.040943 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-83329ba0-915e-4ee2-940a-daf4e4591b5f admin admin] The token cache is being invalidate because project 04ebd3301dbe4be18e75eadc223b2bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:20.041289 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-83329ba0-915e-4ee2-940a-daf4e4591b5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:20.042465 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-83329ba0-915e-4ee2-940a-daf4e4591b5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:20.044684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 911/1823] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:19 2022] DELETE /identity/v3/projects/04ebd3301dbe4be18e75eadc223b2bd3 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:20.051279 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e60b9b-fc65-421b-944a-a5c74b8c1168 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:20.051672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e60b9b-fc65-421b-944a-a5c74b8c1168 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:20.051901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e60b9b-fc65-421b-944a-a5c74b8c1168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:20.096592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-11e60b9b-fc65-421b-944a-a5c74b8c1168 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:20.098463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-11e60b9b-fc65-421b-944a-a5c74b8c1168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:20.184402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 913/1824] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:20.192830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0a855-a4f2-4250-8c54-610465da4253 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:20.193113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0a855-a4f2-4250-8c54-610465da4253 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:20.193390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0a855-a4f2-4250-8c54-610465da4253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:20.216073 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-6ad0a855-a4f2-4250-8c54-610465da4253 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:20.217345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6ad0a855-a4f2-4250-8c54-610465da4253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:20.225425 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 912/1825] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:20.229854 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e60b9b-fc65-421b-944a-a5c74b8c1168 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:20.240504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e60b9b-fc65-421b-944a-a5c74b8c1168 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:20.246164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca4d50-276f-4d1e-831c-49d07764f0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:20.246974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4d50-276f-4d1e-831c-49d07764f0bd admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:20.247358 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4d50-276f-4d1e-831c-49d07764f0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:20.247676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4d50-276f-4d1e-831c-49d07764f0bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:20.248584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca4d50-276f-4d1e-831c-49d07764f0bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:17:20.248946 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca4d50-276f-4d1e-831c-49d07764f0bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:20.251033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca4d50-276f-4d1e-831c-49d07764f0bd admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:20.272530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 914/1826] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:17:20.273306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0a855-a4f2-4250-8c54-610465da4253 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:20.296163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0a855-a4f2-4250-8c54-610465da4253 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:20.308257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ee444-53c3-4c70-873b-071bd0d97223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:20.308910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ee444-53c3-4c70-873b-071bd0d97223 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:20.309182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ee444-53c3-4c70-873b-071bd0d97223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:20.309453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ee444-53c3-4c70-873b-071bd0d97223 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:20.310491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ee444-53c3-4c70-873b-071bd0d97223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:20.312633 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ee444-53c3-4c70-873b-071bd0d97223 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:20.349115 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 913/1827] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:20 2022] POST /identity/v3/projects => generated 387 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:20.355639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca4d50-276f-4d1e-831c-49d07764f0bd admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:20.369722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca4d50-276f-4d1e-831c-49d07764f0bd admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:20.455629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9683bd7-57af-4cee-b48d-fe0d7baed97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:20.456298 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9683bd7-57af-4cee-b48d-fe0d7baed97a admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:20.456534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9683bd7-57af-4cee-b48d-fe0d7baed97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:20.457069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9683bd7-57af-4cee-b48d-fe0d7baed97a admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:20.457680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9683bd7-57af-4cee-b48d-fe0d7baed97a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:20.459676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9683bd7-57af-4cee-b48d-fe0d7baed97a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:20.485232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 915/1828] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:20 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:20.490327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ee444-53c3-4c70-873b-071bd0d97223 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:20.502647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ee444-53c3-4c70-873b-071bd0d97223 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:20.507793 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f18e7e-7f8e-45e8-a6ec-d09319d96b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:20.508546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f18e7e-7f8e-45e8-a6ec-d09319d96b78 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:20.508896 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f18e7e-7f8e-45e8-a6ec-d09319d96b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:20.509238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f18e7e-7f8e-45e8-a6ec-d09319d96b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:20.510513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f18e7e-7f8e-45e8-a6ec-d09319d96b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:20.512384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f18e7e-7f8e-45e8-a6ec-d09319d96b78 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:20.524585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 914/1829] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:20.529420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9683bd7-57af-4cee-b48d-fe0d7baed97a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:20.541096 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9683bd7-57af-4cee-b48d-fe0d7baed97a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:20.546072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496790fe-7290-4374-9503-e7eaa9f9a9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:20.546701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496790fe-7290-4374-9503-e7eaa9f9a9f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:20.546980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496790fe-7290-4374-9503-e7eaa9f9a9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:20.547315 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496790fe-7290-4374-9503-e7eaa9f9a9f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:20.547975 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496790fe-7290-4374-9503-e7eaa9f9a9f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:20.549672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496790fe-7290-4374-9503-e7eaa9f9a9f6 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:20.563047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 916/1830] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:20.568375 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f18e7e-7f8e-45e8-a6ec-d09319d96b78 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:20.585063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f18e7e-7f8e-45e8-a6ec-d09319d96b78 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:20.602805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934133e-e70a-44fb-b32f-c7d2bf79dd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:20.605629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934133e-e70a-44fb-b32f-c7d2bf79dd11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:20.606090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934133e-e70a-44fb-b32f-c7d2bf79dd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:20.606957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934133e-e70a-44fb-b32f-c7d2bf79dd11 admin admin] PATH_INFO: `/v3/projects/a57d0c4d401044b5a30ad43edd8f94e3/users/fefba5183fd4451b98b96def4bf2ccf7/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:20.650172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934133e-e70a-44fb-b32f-c7d2bf79dd11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a57d0c4d401044b5a30ad43edd8f94e3, user_id=fefba5183fd4451b98b96def4bf2ccf7, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:20.653746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934133e-e70a-44fb-b32f-c7d2bf79dd11 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:20.675318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 915/1831] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:20 2022] PUT /identity/v3/projects/a57d0c4d401044b5a30ad43edd8f94e3/users/fefba5183fd4451b98b96def4bf2ccf7/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:20.698722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c4ce2-6ab8-4376-87a4-21b7905977e4 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:20.699277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c4ce2-6ab8-4376-87a4-21b7905977e4 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:20.699772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c4ce2-6ab8-4376-87a4-21b7905977e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:20.750332 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-4a1c4ce2-6ab8-4376-87a4-21b7905977e4 None None] MFA Rules not processed for user `fefba5183fd4451b98b96def4bf2ccf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:20.752101 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4a1c4ce2-6ab8-4376-87a4-21b7905977e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:20.794333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:20.876774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 917/1832] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:20.883248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934133e-e70a-44fb-b32f-c7d2bf79dd11 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:21.018249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934133e-e70a-44fb-b32f-c7d2bf79dd11 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:21.024297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82a9e-826f-433d-806f-ee569a941628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:21.025085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82a9e-826f-433d-806f-ee569a941628 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:21.025917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82a9e-826f-433d-806f-ee569a941628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:21.026215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82a9e-826f-433d-806f-ee569a941628 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:21.026878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af82a9e-826f-433d-806f-ee569a941628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:21.029177 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af82a9e-826f-433d-806f-ee569a941628 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:21.039871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:21.072406 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 916/1833] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:20 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:21.077487 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c4ce2-6ab8-4376-87a4-21b7905977e4 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:21.091631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c4ce2-6ab8-4376-87a4-21b7905977e4 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:21.169726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1bac2-91b5-4b3a-b748-b6acec43b08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:21.170356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1bac2-91b5-4b3a-b748-b6acec43b08b admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:21.171306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1bac2-91b5-4b3a-b748-b6acec43b08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:21.171306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1bac2-91b5-4b3a-b748-b6acec43b08b admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:21.171651 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1bac2-91b5-4b3a-b748-b6acec43b08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:21.174328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1bac2-91b5-4b3a-b748-b6acec43b08b admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:21.206344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 918/1834] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:21 2022] POST /identity/v3/users => generated 300 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:21.207648 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82a9e-826f-433d-806f-ee569a941628 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:21.221230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82a9e-826f-433d-806f-ee569a941628 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:21.229278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c4cec-f691-4f43-b8ad-e3fec7b1cb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:21.230290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c4cec-f691-4f43-b8ad-e3fec7b1cb0d admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:21.230692 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c4cec-f691-4f43-b8ad-e3fec7b1cb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:21.231037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c4cec-f691-4f43-b8ad-e3fec7b1cb0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:21.232139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c4cec-f691-4f43-b8ad-e3fec7b1cb0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:21.234638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c4cec-f691-4f43-b8ad-e3fec7b1cb0d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:21.250369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 917/1835] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:21.251865 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1bac2-91b5-4b3a-b748-b6acec43b08b admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:21.270078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1bac2-91b5-4b3a-b748-b6acec43b08b admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:21.286195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72ffee-e9f0-4183-b2fe-4823712018e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:21.288130 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72ffee-e9f0-4183-b2fe-4823712018e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:21.289415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72ffee-e9f0-4183-b2fe-4823712018e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:21.290049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72ffee-e9f0-4183-b2fe-4823712018e8 admin admin] PATH_INFO: `/v3/projects/462202c3c4e24b40a19594096e8ba458/users/c1657ac63d5245d9b1e97bfaf205f7da/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:21.314026 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72ffee-e9f0-4183-b2fe-4823712018e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=462202c3c4e24b40a19594096e8ba458, user_id=c1657ac63d5245d9b1e97bfaf205f7da, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:21.317434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72ffee-e9f0-4183-b2fe-4823712018e8 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:21.340225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 919/1836] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:21 2022] PUT /identity/v3/projects/462202c3c4e24b40a19594096e8ba458/users/c1657ac63d5245d9b1e97bfaf205f7da/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:21.345516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c4cec-f691-4f43-b8ad-e3fec7b1cb0d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:21.357212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c4cec-f691-4f43-b8ad-e3fec7b1cb0d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:21.474152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621514bb-20a5-4cee-a613-49d9e47d5640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:21.477635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621514bb-20a5-4cee-a613-49d9e47d5640 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:21.478126 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621514bb-20a5-4cee-a613-49d9e47d5640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:21.479204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621514bb-20a5-4cee-a613-49d9e47d5640 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:21.480903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621514bb-20a5-4cee-a613-49d9e47d5640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:21.484387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621514bb-20a5-4cee-a613-49d9e47d5640 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:21.514185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 918/1837] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:21.518964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72ffee-e9f0-4183-b2fe-4823712018e8 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:21.534623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72ffee-e9f0-4183-b2fe-4823712018e8 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:21.543060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f029376-a486-46d0-957a-28cc7c89485b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:21.544785 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f029376-a486-46d0-957a-28cc7c89485b admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:21.545419 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f029376-a486-46d0-957a-28cc7c89485b admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:21.545891 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f029376-a486-46d0-957a-28cc7c89485b admin admin] PATH_INFO: `/v3/projects/462202c3c4e24b40a19594096e8ba458/users/c1657ac63d5245d9b1e97bfaf205f7da/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:21.549384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f029376-a486-46d0-957a-28cc7c89485b admin admin] RBAC: Authorizing `identity:create_grant(project_id=462202c3c4e24b40a19594096e8ba458, user_id=c1657ac63d5245d9b1e97bfaf205f7da, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:21.553894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f029376-a486-46d0-957a-28cc7c89485b admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:21.567349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 920/1838] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:21 2022] PUT /identity/v3/projects/462202c3c4e24b40a19594096e8ba458/users/c1657ac63d5245d9b1e97bfaf205f7da/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:21.581179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ca37d-c6bd-42f9-96e0-a062be5ccaf2 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:21.581817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ca37d-c6bd-42f9-96e0-a062be5ccaf2 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:21.582345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ca37d-c6bd-42f9-96e0-a062be5ccaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:21.626851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-e09ca37d-c6bd-42f9-96e0-a062be5ccaf2 None None] MFA Rules not processed for user `c1657ac63d5245d9b1e97bfaf205f7da`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:21.628161 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e09ca37d-c6bd-42f9-96e0-a062be5ccaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:21.748751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 919/1839] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:21 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:21.763541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f029376-a486-46d0-957a-28cc7c89485b admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:21.766012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-0f029376-a486-46d0-957a-28cc7c89485b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:21.850125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f029376-a486-46d0-957a-28cc7c89485b admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:21.854730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7fe9c-992b-41ad-a092-a1b6617dffec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:21.855417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7fe9c-992b-41ad-a092-a1b6617dffec service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:21.855709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7fe9c-992b-41ad-a092-a1b6617dffec service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:21.855987 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7fe9c-992b-41ad-a092-a1b6617dffec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:21.866638 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7fe9c-992b-41ad-a092-a1b6617dffec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:21.868295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7fe9c-992b-41ad-a092-a1b6617dffec service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:21.880352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 921/1840] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:22.114011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ca37d-c6bd-42f9-96e0-a062be5ccaf2 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:22.198052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ca37d-c6bd-42f9-96e0-a062be5ccaf2 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:22.205297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:22.207006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:22.207492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:22.207939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] PATH_INFO: `/v3/users/fefba5183fd4451b98b96def4bf2ccf7` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:22.209846 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fefba5183fd4451b98b96def4bf2ccf7)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:22.212951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:22.215830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:22.274502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fefba5183fd4451b98b96def4bf2ccf7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:22.285106 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fefba5183fd4451b98b96def4bf2ccf7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:22.287269 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fefba5183fd4451b98b96def4bf2ccf7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:22.288528 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fefba5183fd4451b98b96def4bf2ccf7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:22.312605 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fefba5183fd4451b98b96def4bf2ccf7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:22.338876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 920/1841] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:22 2022] DELETE /identity/v3/users/fefba5183fd4451b98b96def4bf2ccf7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:22.347672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844e3a8-7191-4bae-b6ad-b9ad5beed6d9 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:22.348434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844e3a8-7191-4bae-b6ad-b9ad5beed6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:22.348885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844e3a8-7191-4bae-b6ad-b9ad5beed6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:22.387570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b844e3a8-7191-4bae-b6ad-b9ad5beed6d9 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:22.388941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b844e3a8-7191-4bae-b6ad-b9ad5beed6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:22.405800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:22.495623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 922/1842] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:22.505940 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:22.507552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:22.595752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361c787-c7d5-440a-80bc-3caa7d1484cd admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:22.602191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07513e7d-e5c5-475b-ae3a-c2d5be4cee1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:22.603262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07513e7d-e5c5-475b-ae3a-c2d5be4cee1e service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:22.604337 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07513e7d-e5c5-475b-ae3a-c2d5be4cee1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:22.604797 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07513e7d-e5c5-475b-ae3a-c2d5be4cee1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:22.627453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07513e7d-e5c5-475b-ae3a-c2d5be4cee1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:22.630419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07513e7d-e5c5-475b-ae3a-c2d5be4cee1e service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:22.651614 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 921/1843] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:17:23.425378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844e3a8-7191-4bae-b6ad-b9ad5beed6d9 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:23.427413 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b844e3a8-7191-4bae-b6ad-b9ad5beed6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:23.440710 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844e3a8-7191-4bae-b6ad-b9ad5beed6d9 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:23.446222 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:23.446884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:23.447744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:23.448115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] PATH_INFO: `/v3/projects/a57d0c4d401044b5a30ad43edd8f94e3` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:23.449223 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] RBAC: Authorizing `identity:delete_project(project_id=a57d0c4d401044b5a30ad43edd8f94e3)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:23.452848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:23.540622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a57d0c4d401044b5a30ad43edd8f94e3', 'request_id': 'req-954259fb-fb3a-4abf-98a8-6094c3fdeede'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:23.551884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a57d0c4d401044b5a30ad43edd8f94e3', 'request_id': 'req-954259fb-fb3a-4abf-98a8-6094c3fdeede'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:23.561489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a57d0c4d401044b5a30ad43edd8f94e3', 'request_id': 'req-954259fb-fb3a-4abf-98a8-6094c3fdeede'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:23.578756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] The token cache is being invalidate because project a57d0c4d401044b5a30ad43edd8f94e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:23.579325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:23.580205 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:23.582259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 923/1844] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:23 2022] DELETE /identity/v3/projects/a57d0c4d401044b5a30ad43edd8f94e3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:23.586329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07513e7d-e5c5-475b-ae3a-c2d5be4cee1e service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:23.598829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07513e7d-e5c5-475b-ae3a-c2d5be4cee1e service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:23.697974 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec74c0-86d3-47c5-b896-16c417511cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:23.699040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec74c0-86d3-47c5-b896-16c417511cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:23.700314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec74c0-86d3-47c5-b896-16c417511cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:23.700494 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec74c0-86d3-47c5-b896-16c417511cf2 admin admin] PATH_INFO: `/v3/users/c1657ac63d5245d9b1e97bfaf205f7da` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:23.701878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec74c0-86d3-47c5-b896-16c417511cf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1657ac63d5245d9b1e97bfaf205f7da)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:23.704501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec74c0-86d3-47c5-b896-16c417511cf2 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:23.738071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:23.787455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-03ec74c0-86d3-47c5-b896-16c417511cf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1657ac63d5245d9b1e97bfaf205f7da'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:23.804689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-03ec74c0-86d3-47c5-b896-16c417511cf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1657ac63d5245d9b1e97bfaf205f7da'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:23.805470 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-03ec74c0-86d3-47c5-b896-16c417511cf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1657ac63d5245d9b1e97bfaf205f7da'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:23.806133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-03ec74c0-86d3-47c5-b896-16c417511cf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1657ac63d5245d9b1e97bfaf205f7da'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:23.832076 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-03ec74c0-86d3-47c5-b896-16c417511cf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1657ac63d5245d9b1e97bfaf205f7da'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:23.861246 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 922/1845] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:23 2022] DELETE /identity/v3/users/c1657ac63d5245d9b1e97bfaf205f7da => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:24.266336 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:24.268355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:24.289971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954259fb-fb3a-4abf-98a8-6094c3fdeede admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:24.363660 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:24.364317 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:24.364602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:24.364880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3 admin admin] PATH_INFO: `/v3/projects/462202c3c4e24b40a19594096e8ba458` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:24.366049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=462202c3c4e24b40a19594096e8ba458)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:24.368025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:24.432141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:24.435358 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '462202c3c4e24b40a19594096e8ba458', 'request_id': 'req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:24.443282 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '462202c3c4e24b40a19594096e8ba458', 'request_id': 'req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:24.453726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '462202c3c4e24b40a19594096e8ba458', 'request_id': 'req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:24.473091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3 admin admin] The token cache is being invalidate because project 462202c3c4e24b40a19594096e8ba458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:24.473400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:24.474125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-5cca176a-8e14-4e0f-9ef8-dd9caea9c3b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:24.475800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 924/1846] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:24 2022] DELETE /identity/v3/projects/462202c3c4e24b40a19594096e8ba458 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:24.481120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cb9c6-eea2-4b0a-8ec5-c1c9a880ee07 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:24.481419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cb9c6-eea2-4b0a-8ec5-c1c9a880ee07 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:24.481701 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cb9c6-eea2-4b0a-8ec5-c1c9a880ee07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:24.504720 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-995cb9c6-eea2-4b0a-8ec5-c1c9a880ee07 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:24.505981 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-995cb9c6-eea2-4b0a-8ec5-c1c9a880ee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:24.579315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 923/1847] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:24.588981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a6371-165b-422c-bde5-de698d37ecd7 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:24.589262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a6371-165b-422c-bde5-de698d37ecd7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:24.589748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a6371-165b-422c-bde5-de698d37ecd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:24.611244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-1d2a6371-165b-422c-bde5-de698d37ecd7 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:24.612497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1d2a6371-165b-422c-bde5-de698d37ecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:24.622215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 925/1848] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:24.626830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cb9c6-eea2-4b0a-8ec5-c1c9a880ee07 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:24.637387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cb9c6-eea2-4b0a-8ec5-c1c9a880ee07 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:24.644098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186eded-ebbf-45e8-b4d5-fb3fbfa574c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:24.644098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186eded-ebbf-45e8-b4d5-fb3fbfa574c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:24.644511 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186eded-ebbf-45e8-b4d5-fb3fbfa574c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:24.644840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186eded-ebbf-45e8-b4d5-fb3fbfa574c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:24.645651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186eded-ebbf-45e8-b4d5-fb3fbfa574c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:17:24.646004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186eded-ebbf-45e8-b4d5-fb3fbfa574c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:24.647969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186eded-ebbf-45e8-b4d5-fb3fbfa574c7 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:24.668235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 924/1849] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:17:24.673086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a6371-165b-422c-bde5-de698d37ecd7 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:24.684595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a6371-165b-422c-bde5-de698d37ecd7 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:24.690132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da86bf0-fc92-4172-a410-e08be756e30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:24.691021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da86bf0-fc92-4172-a410-e08be756e30d admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:24.694411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da86bf0-fc92-4172-a410-e08be756e30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:24.696657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da86bf0-fc92-4172-a410-e08be756e30d admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:24.698614 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da86bf0-fc92-4172-a410-e08be756e30d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:24.702791 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da86bf0-fc92-4172-a410-e08be756e30d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:24.746610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 926/1850] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:24 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:17:24.751059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186eded-ebbf-45e8-b4d5-fb3fbfa574c7 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:24.762279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186eded-ebbf-45e8-b4d5-fb3fbfa574c7 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:24.827024 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e2141-35c8-46b1-af15-71f9b1865803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:24.827730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e2141-35c8-46b1-af15-71f9b1865803 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:24.828011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e2141-35c8-46b1-af15-71f9b1865803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:24.828283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e2141-35c8-46b1-af15-71f9b1865803 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:24.828929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e2141-35c8-46b1-af15-71f9b1865803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:24.830896 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e2141-35c8-46b1-af15-71f9b1865803 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:24.872917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 925/1851] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:24 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:24.876037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da86bf0-fc92-4172-a410-e08be756e30d admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:24.895899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da86bf0-fc92-4172-a410-e08be756e30d admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:24.905488 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf2a5c-981b-44f4-a65d-6341fd422a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:24.906525 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf2a5c-981b-44f4-a65d-6341fd422a81 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:24.907001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf2a5c-981b-44f4-a65d-6341fd422a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:24.907514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf2a5c-981b-44f4-a65d-6341fd422a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:24.908654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf2a5c-981b-44f4-a65d-6341fd422a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:24.911820 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf2a5c-981b-44f4-a65d-6341fd422a81 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:24.926512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 927/1852] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:24.931625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e2141-35c8-46b1-af15-71f9b1865803 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:24.948308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e2141-35c8-46b1-af15-71f9b1865803 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:24.954960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc1acb-611a-40ba-9a5f-a56da12cabde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:24.956081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc1acb-611a-40ba-9a5f-a56da12cabde admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:24.956448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc1acb-611a-40ba-9a5f-a56da12cabde admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:24.956787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc1acb-611a-40ba-9a5f-a56da12cabde admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:24.957642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc1acb-611a-40ba-9a5f-a56da12cabde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:24.960526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc1acb-611a-40ba-9a5f-a56da12cabde admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:24.989026 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 926/1853] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:24.995823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf2a5c-981b-44f4-a65d-6341fd422a81 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:25.024248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf2a5c-981b-44f4-a65d-6341fd422a81 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:25.052287 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e026a-cac6-4667-aeb5-5f4a37dad401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:25.053351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e026a-cac6-4667-aeb5-5f4a37dad401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:25.054116 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e026a-cac6-4667-aeb5-5f4a37dad401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:25.054517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e026a-cac6-4667-aeb5-5f4a37dad401 admin admin] PATH_INFO: `/v3/projects/eb09beb570b641bba1031dc5e2458eef/users/932d563291d345748609486cf04755ab/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:25.128098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e026a-cac6-4667-aeb5-5f4a37dad401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb09beb570b641bba1031dc5e2458eef, user_id=932d563291d345748609486cf04755ab, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:25.133419 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e026a-cac6-4667-aeb5-5f4a37dad401 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:25.182793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 928/1854] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:24 2022] PUT /identity/v3/projects/eb09beb570b641bba1031dc5e2458eef/users/932d563291d345748609486cf04755ab/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:25.194559 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c3ad-329a-4370-9eb9-106c6d300fe1 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:25.194909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c3ad-329a-4370-9eb9-106c6d300fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:25.195230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c3ad-329a-4370-9eb9-106c6d300fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:25.228667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:25.279915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-abe6c3ad-329a-4370-9eb9-106c6d300fe1 None None] MFA Rules not processed for user `932d563291d345748609486cf04755ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:25.284403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-abe6c3ad-329a-4370-9eb9-106c6d300fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:25.416597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 927/1855] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:25.421131 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e026a-cac6-4667-aeb5-5f4a37dad401 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:25.522048 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e026a-cac6-4667-aeb5-5f4a37dad401 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:25.534132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611e29a-49c4-4acf-b76f-79702ceadae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:25.535983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611e29a-49c4-4acf-b76f-79702ceadae1 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:25.537000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611e29a-49c4-4acf-b76f-79702ceadae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:25.537739 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611e29a-49c4-4acf-b76f-79702ceadae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:25.538730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611e29a-49c4-4acf-b76f-79702ceadae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:25.544013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611e29a-49c4-4acf-b76f-79702ceadae1 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:25.615676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 929/1856] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:25 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:17:25.616623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6c3ad-329a-4370-9eb9-106c6d300fe1 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:25.630516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6c3ad-329a-4370-9eb9-106c6d300fe1 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:25.730645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f07a3-63d1-4d94-a276-84ff7bbf2391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:25.732207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f07a3-63d1-4d94-a276-84ff7bbf2391 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:25.732744 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f07a3-63d1-4d94-a276-84ff7bbf2391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:25.733504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f07a3-63d1-4d94-a276-84ff7bbf2391 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:25.734910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f07a3-63d1-4d94-a276-84ff7bbf2391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:25.737140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f07a3-63d1-4d94-a276-84ff7bbf2391 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:25.780570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 928/1857] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:25 2022] POST /identity/v3/users => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:25.783033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611e29a-49c4-4acf-b76f-79702ceadae1 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:25.798335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611e29a-49c4-4acf-b76f-79702ceadae1 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:25.812192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcf509-aa8f-4acc-965a-5cdd480e7b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:25.813179 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcf509-aa8f-4acc-965a-5cdd480e7b60 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:25.813669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcf509-aa8f-4acc-965a-5cdd480e7b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:25.814094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcf509-aa8f-4acc-965a-5cdd480e7b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:25.814913 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcf509-aa8f-4acc-965a-5cdd480e7b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:25.816565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcf509-aa8f-4acc-965a-5cdd480e7b60 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:25.839140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 930/1858] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:25.844690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f07a3-63d1-4d94-a276-84ff7bbf2391 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:25.859527 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f07a3-63d1-4d94-a276-84ff7bbf2391 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:25.871050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6ff61-ba42-45df-a326-1de060879675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:25.871873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6ff61-ba42-45df-a326-1de060879675 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:25.872412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6ff61-ba42-45df-a326-1de060879675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:25.872756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6ff61-ba42-45df-a326-1de060879675 admin admin] PATH_INFO: `/v3/projects/fa6f8af2c4f44b7d8ec842e60b88e341/users/83eae1eaf91043febe5bd82e92b27047/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:25.894952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6ff61-ba42-45df-a326-1de060879675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa6f8af2c4f44b7d8ec842e60b88e341, user_id=83eae1eaf91043febe5bd82e92b27047, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:25.904657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6ff61-ba42-45df-a326-1de060879675 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:25.929412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 929/1859] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:25 2022] PUT /identity/v3/projects/fa6f8af2c4f44b7d8ec842e60b88e341/users/83eae1eaf91043febe5bd82e92b27047/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:25.933554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcf509-aa8f-4acc-965a-5cdd480e7b60 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:25.953083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcf509-aa8f-4acc-965a-5cdd480e7b60 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:26.052905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01142969-6bad-4525-b1f4-975a9a9b2f18 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:26.053287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01142969-6bad-4525-b1f4-975a9a9b2f18 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:26.053501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01142969-6bad-4525-b1f4-975a9a9b2f18 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:26.058295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 930/1860] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:17:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:17:26.080932 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6fa5b-abef-4817-9ee1-b6bb14e991fb None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:26.081280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6fa5b-abef-4817-9ee1-b6bb14e991fb None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:26.082441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6fa5b-abef-4817-9ee1-b6bb14e991fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:26.117749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:26.129993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-6da6fa5b-abef-4817-9ee1-b6bb14e991fb None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:26.135642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6da6fa5b-abef-4817-9ee1-b6bb14e991fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:26.162388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf84842-da2c-4435-889e-e79cecd83e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:26.165360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf84842-da2c-4435-889e-e79cecd83e53 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:26.165806 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf84842-da2c-4435-889e-e79cecd83e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:26.166303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf84842-da2c-4435-889e-e79cecd83e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:26.167510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf84842-da2c-4435-889e-e79cecd83e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:26.170736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf84842-da2c-4435-889e-e79cecd83e53 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:26.199971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 931/1861] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:26.201327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf84842-da2c-4435-889e-e79cecd83e53 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:26.225969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf84842-da2c-4435-889e-e79cecd83e53 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:26.238388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2862ca-20b8-4f80-a627-3cae09e37321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:26.239010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2862ca-20b8-4f80-a627-3cae09e37321 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:26.239656 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2862ca-20b8-4f80-a627-3cae09e37321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:26.240511 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2862ca-20b8-4f80-a627-3cae09e37321 admin admin] PATH_INFO: `/v3/projects/fa6f8af2c4f44b7d8ec842e60b88e341/users/83eae1eaf91043febe5bd82e92b27047/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:26.246294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2862ca-20b8-4f80-a627-3cae09e37321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa6f8af2c4f44b7d8ec842e60b88e341, user_id=83eae1eaf91043febe5bd82e92b27047, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:26.254455 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2862ca-20b8-4f80-a627-3cae09e37321 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:26.281044 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 932/1862] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:26 2022] PUT /identity/v3/projects/fa6f8af2c4f44b7d8ec842e60b88e341/users/83eae1eaf91043febe5bd82e92b27047/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:26.293196 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a707f8-8e9f-4a33-9028-98d14512fcb2 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:26.293725 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a707f8-8e9f-4a33-9028-98d14512fcb2 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:26.294133 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a707f8-8e9f-4a33-9028-98d14512fcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:26.319324 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:26.366402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 931/1863] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:17:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:26.422951 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-c8a707f8-8e9f-4a33-9028-98d14512fcb2 None None] MFA Rules not processed for user `83eae1eaf91043febe5bd82e92b27047`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:26.426184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c8a707f8-8e9f-4a33-9028-98d14512fcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:26.433328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6fa5b-abef-4817-9ee1-b6bb14e991fb None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:26.437096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6da6fa5b-abef-4817-9ee1-b6bb14e991fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:26.470494 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6fa5b-abef-4817-9ee1-b6bb14e991fb None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:26.640596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 933/1864] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:26.644311 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a886444-9b46-4df5-9064-84eff336264d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:26.645047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a886444-9b46-4df5-9064-84eff336264d service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:26.645368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a886444-9b46-4df5-9064-84eff336264d service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:26.645668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a886444-9b46-4df5-9064-84eff336264d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:26.655591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a707f8-8e9f-4a33-9028-98d14512fcb2 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:26.657300 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c8a707f8-8e9f-4a33-9028-98d14512fcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:26.659911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a886444-9b46-4df5-9064-84eff336264d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:26.663292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a886444-9b46-4df5-9064-84eff336264d service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:26.688529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 932/1865] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:26.753036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a707f8-8e9f-4a33-9028-98d14512fcb2 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:26.758168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82bf1b-edb5-48aa-ba89-12afc2edc99a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:26.759098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82bf1b-edb5-48aa-ba89-12afc2edc99a service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:26.760245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82bf1b-edb5-48aa-ba89-12afc2edc99a service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:26.760619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82bf1b-edb5-48aa-ba89-12afc2edc99a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:26.788342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82bf1b-edb5-48aa-ba89-12afc2edc99a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:26.790171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82bf1b-edb5-48aa-ba89-12afc2edc99a service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:26.807731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 934/1866] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:26.985113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a886444-9b46-4df5-9064-84eff336264d service nova] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:27.010101 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a886444-9b46-4df5-9064-84eff336264d service nova] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:27.021133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917dc4a-9d9a-441e-aa68-ec1eeca2c6a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:27.021760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917dc4a-9d9a-441e-aa68-ec1eeca2c6a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:27.022036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917dc4a-9d9a-441e-aa68-ec1eeca2c6a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:27.022535 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917dc4a-9d9a-441e-aa68-ec1eeca2c6a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:27.132115 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917dc4a-9d9a-441e-aa68-ec1eeca2c6a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:27.133686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917dc4a-9d9a-441e-aa68-ec1eeca2c6a4 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:27.146341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 933/1867] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:27.418737 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82bf1b-edb5-48aa-ba89-12afc2edc99a service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:27.420277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ae82bf1b-edb5-48aa-ba89-12afc2edc99a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:27.568559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:27.569988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917dc4a-9d9a-441e-aa68-ec1eeca2c6a4 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:27.576823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82bf1b-edb5-48aa-ba89-12afc2edc99a service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:27.588271 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917dc4a-9d9a-441e-aa68-ec1eeca2c6a4 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:27.588664 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298eafb-cc03-44f1-b7a5-333d571638f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:27.589502 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298eafb-cc03-44f1-b7a5-333d571638f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:27.589940 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298eafb-cc03-44f1-b7a5-333d571638f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:27.590370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298eafb-cc03-44f1-b7a5-333d571638f8 admin admin] PATH_INFO: `/v3/users/c936ea7e2f224d03926a2004476dff5a` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:27.592142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298eafb-cc03-44f1-b7a5-333d571638f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c936ea7e2f224d03926a2004476dff5a)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:27.593799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed09017-2ceb-40bb-b898-097a0be98260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:27.594456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed09017-2ceb-40bb-b898-097a0be98260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:27.594739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed09017-2ceb-40bb-b898-097a0be98260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:27.595049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298eafb-cc03-44f1-b7a5-333d571638f8 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:27.595202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed09017-2ceb-40bb-b898-097a0be98260 admin admin] PATH_INFO: `/v3/users/932d563291d345748609486cf04755ab` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:27.596322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed09017-2ceb-40bb-b898-097a0be98260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=932d563291d345748609486cf04755ab)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:27.598613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed09017-2ceb-40bb-b898-097a0be98260 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:27.670564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4298eafb-cc03-44f1-b7a5-333d571638f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c936ea7e2f224d03926a2004476dff5a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:27.675404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-eed09017-2ceb-40bb-b898-097a0be98260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '932d563291d345748609486cf04755ab'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:27.689604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-eed09017-2ceb-40bb-b898-097a0be98260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '932d563291d345748609486cf04755ab'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:27.690861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-eed09017-2ceb-40bb-b898-097a0be98260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '932d563291d345748609486cf04755ab'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:27.692004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-eed09017-2ceb-40bb-b898-097a0be98260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '932d563291d345748609486cf04755ab'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:27.692452 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4298eafb-cc03-44f1-b7a5-333d571638f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c936ea7e2f224d03926a2004476dff5a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:27.704312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4298eafb-cc03-44f1-b7a5-333d571638f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c936ea7e2f224d03926a2004476dff5a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:27.705601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4298eafb-cc03-44f1-b7a5-333d571638f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c936ea7e2f224d03926a2004476dff5a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:27.706352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4298eafb-cc03-44f1-b7a5-333d571638f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c936ea7e2f224d03926a2004476dff5a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:27.715622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-eed09017-2ceb-40bb-b898-097a0be98260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '932d563291d345748609486cf04755ab'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:27.719629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:27.738635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 934/1868] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:27 2022] DELETE /identity/v3/users/932d563291d345748609486cf04755ab => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:27.742907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3faecd-e25a-44d2-bd86-680688e3f1ac None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:27.744363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3faecd-e25a-44d2-bd86-680688e3f1ac None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:27.745100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3faecd-e25a-44d2-bd86-680688e3f1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:27.750901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 935/1869] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:27 2022] DELETE /identity/v3/users/c936ea7e2f224d03926a2004476dff5a => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:27.764059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fb1b8-c08a-4e3e-bf72-2482bbe68411 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:27.764633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fb1b8-c08a-4e3e-bf72-2482bbe68411 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:27.765108 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fb1b8-c08a-4e3e-bf72-2482bbe68411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:27.782331 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-6d3faecd-e25a-44d2-bd86-680688e3f1ac None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:27.783970 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6d3faecd-e25a-44d2-bd86-680688e3f1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:27.792310 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-541fb1b8-c08a-4e3e-bf72-2482bbe68411 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:27.793789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-541fb1b8-c08a-4e3e-bf72-2482bbe68411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:27.920331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 936/1870] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:27.922136 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fb1b8-c08a-4e3e-bf72-2482bbe68411 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:27.924476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-541fb1b8-c08a-4e3e-bf72-2482bbe68411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:27.940321 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fb1b8-c08a-4e3e-bf72-2482bbe68411 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:27.951991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:27.952653 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:27.952933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:27.953263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] PATH_INFO: `/v3/users/65ec656d73d846f489ea4bdd877e058d` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:27.958764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65ec656d73d846f489ea4bdd877e058d)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:27.965745 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 935/1871] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:27.967320 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3faecd-e25a-44d2-bd86-680688e3f1ac None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:27.969215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6d3faecd-e25a-44d2-bd86-680688e3f1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:27.969777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:28.088400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65ec656d73d846f489ea4bdd877e058d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:28.128071 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65ec656d73d846f489ea4bdd877e058d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:28.145756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:28.171314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3faecd-e25a-44d2-bd86-680688e3f1ac None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:28.173714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65ec656d73d846f489ea4bdd877e058d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:28.173714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65ec656d73d846f489ea4bdd877e058d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:28.173714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65ec656d73d846f489ea4bdd877e058d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:28.185043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b255dd-6a0c-401c-8236-6ddbf87f3e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:28.187249 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b255dd-6a0c-401c-8236-6ddbf87f3e22 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:28.188262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b255dd-6a0c-401c-8236-6ddbf87f3e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:28.189212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b255dd-6a0c-401c-8236-6ddbf87f3e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:28.192719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-52b255dd-6a0c-401c-8236-6ddbf87f3e22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:28.200518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 937/1872] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:27 2022] DELETE /identity/v3/users/65ec656d73d846f489ea4bdd877e058d => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:28.201390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:28.205196 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:28.223591 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b255dd-6a0c-401c-8236-6ddbf87f3e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:28.227686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b255dd-6a0c-401c-8236-6ddbf87f3e22 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:28.258757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 936/1873] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:28.263021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51de898-1e5d-462c-86bc-1eb30a662227 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:28.263788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51de898-1e5d-462c-86bc-1eb30a662227 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:28.264626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51de898-1e5d-462c-86bc-1eb30a662227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:28.324163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-b51de898-1e5d-462c-86bc-1eb30a662227 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:28.332543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b51de898-1e5d-462c-86bc-1eb30a662227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:28.383633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edd0bc-28e8-46e9-8e61-0652bd3d8265 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:28.390583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d1eb5-3d20-4325-8c54-6155e01bc8e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:28.391779 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d1eb5-3d20-4325-8c54-6155e01bc8e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:28.392275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d1eb5-3d20-4325-8c54-6155e01bc8e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:28.392803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d1eb5-3d20-4325-8c54-6155e01bc8e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:28.394858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-083d1eb5-3d20-4325-8c54-6155e01bc8e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:28.506122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 937/1874] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:28.510731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d1eb5-3d20-4325-8c54-6155e01bc8e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:28.514260 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d1eb5-3d20-4325-8c54-6155e01bc8e7 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:28.525083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51de898-1e5d-462c-86bc-1eb30a662227 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:28.542535 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 938/1875] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:28.543752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51de898-1e5d-462c-86bc-1eb30a662227 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:28.548440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2108f1e-4416-4d84-816d-fd3d831570f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:28.550080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2108f1e-4416-4d84-816d-fd3d831570f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:28.550621 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2108f1e-4416-4d84-816d-fd3d831570f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:28.551076 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2108f1e-4416-4d84-816d-fd3d831570f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:28.564344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2108f1e-4416-4d84-816d-fd3d831570f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:28.567092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2108f1e-4416-4d84-816d-fd3d831570f4 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:28.582965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 938/1876] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:29.114935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d1eb5-3d20-4325-8c54-6155e01bc8e7 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:29.116744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-083d1eb5-3d20-4325-8c54-6155e01bc8e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:29.129650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d1eb5-3d20-4325-8c54-6155e01bc8e7 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:29.145179 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:29.146420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:29.147078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:29.147562 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] PATH_INFO: `/v3/projects/ddc95f1674564cb7b1d3297a67a85d02` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:29.148996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddc95f1674564cb7b1d3297a67a85d02)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:29.151772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:29.224295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddc95f1674564cb7b1d3297a67a85d02', 'request_id': 'req-fc67c412-f7da-49e1-926d-358f19d9e4fa'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.237365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddc95f1674564cb7b1d3297a67a85d02', 'request_id': 'req-fc67c412-f7da-49e1-926d-358f19d9e4fa'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.244318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2108f1e-4416-4d84-816d-fd3d831570f4 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:29.246019 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c2108f1e-4416-4d84-816d-fd3d831570f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:29.247298 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddc95f1674564cb7b1d3297a67a85d02', 'request_id': 'req-fc67c412-f7da-49e1-926d-358f19d9e4fa'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.266995 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] The token cache is being invalidate because project ddc95f1674564cb7b1d3297a67a85d02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:29.267358 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.267710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2108f1e-4416-4d84-816d-fd3d831570f4 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:29.268034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.270671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 939/1877] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:29 2022] DELETE /identity/v3/projects/ddc95f1674564cb7b1d3297a67a85d02 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:29.280821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:29.288106 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:29.304130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:29.335971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:29.432736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:29.433561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:29.433844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:29.434117 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] PATH_INFO: `/v3/projects/494c4cf5138f413cb07edab3887ad896` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:29.436805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=494c4cf5138f413cb07edab3887ad896)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:29.440355 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:29.444600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67c412-f7da-49e1-926d-358f19d9e4fa admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:29.450573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:29.451272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:29.451554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:29.451825 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] PATH_INFO: `/v3/users/a7f1edf1fa6544079804e47c2c1939d2` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:29.453058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7f1edf1fa6544079804e47c2c1939d2)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:29.455284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:29.538187 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7f1edf1fa6544079804e47c2c1939d2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.550488 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '494c4cf5138f413cb07edab3887ad896', 'request_id': 'req-ebabc710-58c6-49e0-ac64-ff33ab930bf8'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.555935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7f1edf1fa6544079804e47c2c1939d2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.560167 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '494c4cf5138f413cb07edab3887ad896', 'request_id': 'req-ebabc710-58c6-49e0-ac64-ff33ab930bf8'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.564616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7f1edf1fa6544079804e47c2c1939d2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.565382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7f1edf1fa6544079804e47c2c1939d2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.566977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7f1edf1fa6544079804e47c2c1939d2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.568342 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '494c4cf5138f413cb07edab3887ad896', 'request_id': 'req-ebabc710-58c6-49e0-ac64-ff33ab930bf8'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.583209 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 940/1878] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:29 2022] DELETE /identity/v3/users/a7f1edf1fa6544079804e47c2c1939d2 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:29.584084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:29.585427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:29.590025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] The token cache is being invalidate because project 494c4cf5138f413cb07edab3887ad896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:29.590561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.591747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.594136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 939/1879] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:29 2022] DELETE /identity/v3/projects/494c4cf5138f413cb07edab3887ad896 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:29.601011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:29.603323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:29.703264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6546b23-613c-4ca4-8832-f86e6e7dff86 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:29.711730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:29.715516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:29.715841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:29.716330 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] PATH_INFO: `/v3/projects/eb09beb570b641bba1031dc5e2458eef` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:29.718400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb09beb570b641bba1031dc5e2458eef)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:29.720353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:29.729308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabc710-58c6-49e0-ac64-ff33ab930bf8 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:29.743517 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:29.744856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:29.745310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:29.745592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] PATH_INFO: `/v3/users/4941088d4eea469ea5b0ead03f340b2b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:29.747463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4941088d4eea469ea5b0ead03f340b2b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:29.750273 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:29.809922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:29.821254 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb09beb570b641bba1031dc5e2458eef', 'request_id': 'req-2972e0f9-3365-41ac-8ef7-418b57052fca'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.830270 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb09beb570b641bba1031dc5e2458eef', 'request_id': 'req-2972e0f9-3365-41ac-8ef7-418b57052fca'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.840970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb09beb570b641bba1031dc5e2458eef', 'request_id': 'req-2972e0f9-3365-41ac-8ef7-418b57052fca'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.845099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4941088d4eea469ea5b0ead03f340b2b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.849216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:29.855523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4941088d4eea469ea5b0ead03f340b2b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.856610 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4941088d4eea469ea5b0ead03f340b2b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.857842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4941088d4eea469ea5b0ead03f340b2b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.871861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] The token cache is being invalidate because project eb09beb570b641bba1031dc5e2458eef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:29.872262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.873451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.876181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4941088d4eea469ea5b0ead03f340b2b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:29.876746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 941/1880] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:29 2022] DELETE /identity/v3/projects/eb09beb570b641bba1031dc5e2458eef => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:29.877746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:29.879585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:29.906504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 940/1881] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:29 2022] DELETE /identity/v3/users/4941088d4eea469ea5b0ead03f340b2b => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:29.907880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:29.909483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:29.995201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972e0f9-3365-41ac-8ef7-418b57052fca admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:30.028179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3c770-1f99-4926-8468-5fcf52e23090 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:30.035210 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e511784-9164-47f5-821d-f0fb2c5c28ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:30.036530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e511784-9164-47f5-821d-f0fb2c5c28ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:30.036918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e511784-9164-47f5-821d-f0fb2c5c28ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:30.037294 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e511784-9164-47f5-821d-f0fb2c5c28ef admin admin] PATH_INFO: `/v3/users/83eae1eaf91043febe5bd82e92b27047` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:30.038723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e511784-9164-47f5-821d-f0fb2c5c28ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=83eae1eaf91043febe5bd82e92b27047)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:30.042525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e511784-9164-47f5-821d-f0fb2c5c28ef admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:30.091942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:30.091942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:30.092159 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:30.092371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] PATH_INFO: `/v3/users/c8fde53a9f56451ca82d3b5b3450619c` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:30.094495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8fde53a9f56451ca82d3b5b3450619c)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:30.097616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:30.147296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5e511784-9164-47f5-821d-f0fb2c5c28ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83eae1eaf91043febe5bd82e92b27047'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.159939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5e511784-9164-47f5-821d-f0fb2c5c28ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83eae1eaf91043febe5bd82e92b27047'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.161056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5e511784-9164-47f5-821d-f0fb2c5c28ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83eae1eaf91043febe5bd82e92b27047'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.162040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5e511784-9164-47f5-821d-f0fb2c5c28ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83eae1eaf91043febe5bd82e92b27047'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.183754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5e511784-9164-47f5-821d-f0fb2c5c28ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83eae1eaf91043febe5bd82e92b27047'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.211916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 941/1882] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:29 2022] DELETE /identity/v3/users/83eae1eaf91043febe5bd82e92b27047 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:30.219636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8fde53a9f56451ca82d3b5b3450619c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.234489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8fde53a9f56451ca82d3b5b3450619c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.242591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8fde53a9f56451ca82d3b5b3450619c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.243597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8fde53a9f56451ca82d3b5b3450619c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.244259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8fde53a9f56451ca82d3b5b3450619c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.248911 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:30.261252 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 942/1883] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:29 2022] DELETE /identity/v3/users/c8fde53a9f56451ca82d3b5b3450619c => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:30.266222 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac40b1-0f16-40d4-b2f2-fc029d8a4add None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:30.266869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac40b1-0f16-40d4-b2f2-fc029d8a4add None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:30.267270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac40b1-0f16-40d4-b2f2-fc029d8a4add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:30.336040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-caac40b1-0f16-40d4-b2f2-fc029d8a4add None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:30.338254 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-caac40b1-0f16-40d4-b2f2-fc029d8a4add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:30.431825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:30.463283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 942/1884] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:30.470690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:30.473809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:30.540927 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac40b1-0f16-40d4-b2f2-fc029d8a4add None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:30.544036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-caac40b1-0f16-40d4-b2f2-fc029d8a4add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:30.564540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70edd9-04d0-434a-a3d5-f63b03a1134f admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:30.568179 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab50e5-395f-4225-a74b-00e1c53c6fef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:30.569577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab50e5-395f-4225-a74b-00e1c53c6fef service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:30.569577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab50e5-395f-4225-a74b-00e1c53c6fef service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:30.569577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab50e5-395f-4225-a74b-00e1c53c6fef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:30.583402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab50e5-395f-4225-a74b-00e1c53c6fef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:30.584986 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab50e5-395f-4225-a74b-00e1c53c6fef service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:30.595325 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac40b1-0f16-40d4-b2f2-fc029d8a4add None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:30.597149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 943/1885] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:30.603772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:30.604504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:30.604861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:30.605045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] PATH_INFO: `/v3/projects/43a3b8964d994776817aa8528658714b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:30.607088 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=43a3b8964d994776817aa8528658714b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:30.609647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:30.683660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43a3b8964d994776817aa8528658714b', 'request_id': 'req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.718038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43a3b8964d994776817aa8528658714b', 'request_id': 'req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.725772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43a3b8964d994776817aa8528658714b', 'request_id': 'req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.742667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] The token cache is being invalidate because project 43a3b8964d994776817aa8528658714b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:30.742968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.743874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:30.747990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 943/1886] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:30 2022] DELETE /identity/v3/projects/43a3b8964d994776817aa8528658714b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:30.753494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55d33a-ef19-42a6-a563-bda0adb8149b None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:30.754089 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55d33a-ef19-42a6-a563-bda0adb8149b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:30.754464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55d33a-ef19-42a6-a563-bda0adb8149b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:30.791492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-7a55d33a-ef19-42a6-a563-bda0adb8149b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:30.795521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7a55d33a-ef19-42a6-a563-bda0adb8149b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:30.828679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:30.831541 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:30.887063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 944/1887] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:30.895434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256005be-7839-4d34-af35-6b42c17551f4 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:30.895673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256005be-7839-4d34-af35-6b42c17551f4 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:30.896161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256005be-7839-4d34-af35-6b42c17551f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:30.931145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-256005be-7839-4d34-af35-6b42c17551f4 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:30.932839 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-256005be-7839-4d34-af35-6b42c17551f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:30.949128 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 945/1888] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:30.953686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256005be-7839-4d34-af35-6b42c17551f4 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:30.964529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256005be-7839-4d34-af35-6b42c17551f4 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:30.970067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce84d94-a8f8-490b-ab4c-fa8d5c779c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:30.970707 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce84d94-a8f8-490b-ab4c-fa8d5c779c7a admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:30.970989 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce84d94-a8f8-490b-ab4c-fa8d5c779c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:30.971309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce84d94-a8f8-490b-ab4c-fa8d5c779c7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:30.972087 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce84d94-a8f8-490b-ab4c-fa8d5c779c7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:17:30.972610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce84d94-a8f8-490b-ab4c-fa8d5c779c7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:30.974363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce84d94-a8f8-490b-ab4c-fa8d5c779c7a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:30.982769 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c18d1-b936-4d40-a7fb-85b747f85f3f admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:30.989796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 946/1889] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:17:30.990319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce84d94-a8f8-490b-ab4c-fa8d5c779c7a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:30.992402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-0ce84d94-a8f8-490b-ab4c-fa8d5c779c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:30.993303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:30.994041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:30.994313 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:30.994585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] PATH_INFO: `/v3/projects/fa6f8af2c4f44b7d8ec842e60b88e341` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:30.996964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa6f8af2c4f44b7d8ec842e60b88e341)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:30.998916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:31.016369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce84d94-a8f8-490b-ab4c-fa8d5c779c7a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:31.022506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f76e70-07b3-406b-9794-76dac13ed9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:31.023381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f76e70-07b3-406b-9794-76dac13ed9c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.023496 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f76e70-07b3-406b-9794-76dac13ed9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.023777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f76e70-07b3-406b-9794-76dac13ed9c1 admin admin] PATH_INFO: `/v3/projects/2d1b30f4841e456f822629d36233c075` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.025571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f76e70-07b3-406b-9794-76dac13ed9c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d1b30f4841e456f822629d36233c075)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:31.027696 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f76e70-07b3-406b-9794-76dac13ed9c1 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:31.072137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa6f8af2c4f44b7d8ec842e60b88e341', 'request_id': 'req-8e96b7ae-ff58-459a-a3df-64a698e40ace'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.083404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa6f8af2c4f44b7d8ec842e60b88e341', 'request_id': 'req-8e96b7ae-ff58-459a-a3df-64a698e40ace'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.093693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa6f8af2c4f44b7d8ec842e60b88e341', 'request_id': 'req-8e96b7ae-ff58-459a-a3df-64a698e40ace'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.101505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-60f76e70-07b3-406b-9794-76dac13ed9c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d1b30f4841e456f822629d36233c075', 'request_id': 'req-60f76e70-07b3-406b-9794-76dac13ed9c1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.107709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] The token cache is being invalidate because project fa6f8af2c4f44b7d8ec842e60b88e341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:31.107987 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.108963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.110662 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 944/1890] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:30 2022] DELETE /identity/v3/projects/fa6f8af2c4f44b7d8ec842e60b88e341 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:31.111557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:31.111670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-60f76e70-07b3-406b-9794-76dac13ed9c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d1b30f4841e456f822629d36233c075', 'request_id': 'req-60f76e70-07b3-406b-9794-76dac13ed9c1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.112888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:31.118334 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:31.118741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-60f76e70-07b3-406b-9794-76dac13ed9c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d1b30f4841e456f822629d36233c075', 'request_id': 'req-60f76e70-07b3-406b-9794-76dac13ed9c1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.133773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-60f76e70-07b3-406b-9794-76dac13ed9c1 admin admin] The token cache is being invalidate because project 2d1b30f4841e456f822629d36233c075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:31.134933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-60f76e70-07b3-406b-9794-76dac13ed9c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.135447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-60f76e70-07b3-406b-9794-76dac13ed9c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.141234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 947/1891] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:30 2022] DELETE /identity/v3/projects/2d1b30f4841e456f822629d36233c075 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:31.141234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e60b26-8234-4d36-8604-97711e405c40 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.141234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e60b26-8234-4d36-8604-97711e405c40 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.141234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e60b26-8234-4d36-8604-97711e405c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.149715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:31.162655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-81e60b26-8234-4d36-8604-97711e405c40 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:31.164120 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-81e60b26-8234-4d36-8604-97711e405c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:31.215647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96b7ae-ff58-459a-a3df-64a698e40ace admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:31.302157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca5dab-a153-4078-a3c8-ddabc3a59819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:31.302769 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca5dab-a153-4078-a3c8-ddabc3a59819 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.303008 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca5dab-a153-4078-a3c8-ddabc3a59819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.303271 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca5dab-a153-4078-a3c8-ddabc3a59819 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.303912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca5dab-a153-4078-a3c8-ddabc3a59819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:31.307204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca5dab-a153-4078-a3c8-ddabc3a59819 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:31.311221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 948/1892] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:31.315268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3400f-b8f5-401d-9f95-785e9189ea6d None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.315792 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3400f-b8f5-401d-9f95-785e9189ea6d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.316294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3400f-b8f5-401d-9f95-785e9189ea6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.342540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 945/1893] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:31 2022] POST /identity/v3/projects => generated 373 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:31.344351 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca5dab-a153-4078-a3c8-ddabc3a59819 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:31.346850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-32ca5dab-a153-4078-a3c8-ddabc3a59819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:31.355954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f2d3400f-b8f5-401d-9f95-785e9189ea6d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:31.358064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f2d3400f-b8f5-401d-9f95-785e9189ea6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:31.461134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 949/1894] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:31.463760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4caa8-9eaa-4e5c-b7c5-6bad7944675b None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.464309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4caa8-9eaa-4e5c-b7c5-6bad7944675b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.464798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4caa8-9eaa-4e5c-b7c5-6bad7944675b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.487027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca5dab-a153-4078-a3c8-ddabc3a59819 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:31.494176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-bfe4caa8-9eaa-4e5c-b7c5-6bad7944675b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:31.494842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:31.495661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.495939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.496209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] PATH_INFO: `/v3/projects/e4fe2d2e29b54244b9233b50de580d59` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.497261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bfe4caa8-9eaa-4e5c-b7c5-6bad7944675b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:31.497493 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4fe2d2e29b54244b9233b50de580d59)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:31.499760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:31.507046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 950/1895] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:31.508559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4caa8-9eaa-4e5c-b7c5-6bad7944675b None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:31.510069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bfe4caa8-9eaa-4e5c-b7c5-6bad7944675b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:31.523725 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4caa8-9eaa-4e5c-b7c5-6bad7944675b None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:31.529556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7731c8e-33cf-41b2-9054-ee9db4750004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:31.530283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7731c8e-33cf-41b2-9054-ee9db4750004 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.530661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7731c8e-33cf-41b2-9054-ee9db4750004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.531000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7731c8e-33cf-41b2-9054-ee9db4750004 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.531913 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7731c8e-33cf-41b2-9054-ee9db4750004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:31.534041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7731c8e-33cf-41b2-9054-ee9db4750004 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:31.557814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:31.564598 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 951/1896] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:31 2022] POST /identity/v3/users => generated 287 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:31.566575 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b4cac-f6d1-4ca0-905c-483d2ca3603d None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.567078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b4cac-f6d1-4ca0-905c-483d2ca3603d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.567436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b4cac-f6d1-4ca0-905c-483d2ca3603d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.587907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4fe2d2e29b54244b9233b50de580d59', 'request_id': 'req-a16cf474-e329-47a3-8a74-fa2f2497320e'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.590627 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-520b4cac-f6d1-4ca0-905c-483d2ca3603d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:31.591907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-520b4cac-f6d1-4ca0-905c-483d2ca3603d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:31.598962 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4fe2d2e29b54244b9233b50de580d59', 'request_id': 'req-a16cf474-e329-47a3-8a74-fa2f2497320e'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.607259 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4fe2d2e29b54244b9233b50de580d59', 'request_id': 'req-a16cf474-e329-47a3-8a74-fa2f2497320e'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.627610 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] The token cache is being invalidate because project e4fe2d2e29b54244b9233b50de580d59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:31.627912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.637305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.637305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 946/1897] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:31 2022] DELETE /identity/v3/projects/e4fe2d2e29b54244b9233b50de580d59 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:31.637305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:31.638546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:31.663086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 952/1898] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:31.663086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b4cac-f6d1-4ca0-905c-483d2ca3603d None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:31.663086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-520b4cac-f6d1-4ca0-905c-483d2ca3603d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:31.674370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b4cac-f6d1-4ca0-905c-483d2ca3603d None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:31.681317 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb64d21-8564-4976-a678-74c699be8354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:31.682263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb64d21-8564-4976-a678-74c699be8354 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.682718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb64d21-8564-4976-a678-74c699be8354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.683182 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb64d21-8564-4976-a678-74c699be8354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.684227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb64d21-8564-4976-a678-74c699be8354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:31.686924 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb64d21-8564-4976-a678-74c699be8354 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:31.699736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 953/1899] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:31.700602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb64d21-8564-4976-a678-74c699be8354 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:31.702795 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-8cb64d21-8564-4976-a678-74c699be8354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:31.717359 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb64d21-8564-4976-a678-74c699be8354 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:31.720743 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cf474-e329-47a3-8a74-fa2f2497320e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:31.725124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:31.725667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.725956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.726055 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd6fd7-9a37-4d97-a797-620f8604b80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:31.726304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] PATH_INFO: `/v3/users/c225d229a13f4c2395c359dc23b9a1c2` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.726809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd6fd7-9a37-4d97-a797-620f8604b80a admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.727095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd6fd7-9a37-4d97-a797-620f8604b80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.727559 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd6fd7-9a37-4d97-a797-620f8604b80a admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.727660 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c225d229a13f4c2395c359dc23b9a1c2)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:31.728274 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd6fd7-9a37-4d97-a797-620f8604b80a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:17:31.728576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd6fd7-9a37-4d97-a797-620f8604b80a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:31.729534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:31.730495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd6fd7-9a37-4d97-a797-620f8604b80a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:31.732631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:31.747001 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 947/1900] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:17:31.752240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd6fd7-9a37-4d97-a797-620f8604b80a admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:31.763477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd6fd7-9a37-4d97-a797-620f8604b80a admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:31.768513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a5239-a4c6-4692-982d-cfe599a505aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:31.769163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a5239-a4c6-4692-982d-cfe599a505aa admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.769419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a5239-a4c6-4692-982d-cfe599a505aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.769679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a5239-a4c6-4692-982d-cfe599a505aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.770376 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a5239-a4c6-4692-982d-cfe599a505aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:17:31.770667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a5239-a4c6-4692-982d-cfe599a505aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:31.772334 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a5239-a4c6-4692-982d-cfe599a505aa admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:31.790563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 948/1901] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:17:31.792291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a5239-a4c6-4692-982d-cfe599a505aa admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:31.793685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c225d229a13f4c2395c359dc23b9a1c2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.803956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a5239-a4c6-4692-982d-cfe599a505aa admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:31.809969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b1ac8-6ba5-4951-88ef-f6f897165b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:31.810547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b1ac8-6ba5-4951-88ef-f6f897165b1d admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.810831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b1ac8-6ba5-4951-88ef-f6f897165b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.811365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b1ac8-6ba5-4951-88ef-f6f897165b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.812557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b1ac8-6ba5-4951-88ef-f6f897165b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:31.813275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c225d229a13f4c2395c359dc23b9a1c2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.820341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b1ac8-6ba5-4951-88ef-f6f897165b1d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:31.822629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c225d229a13f4c2395c359dc23b9a1c2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.823923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c225d229a13f4c2395c359dc23b9a1c2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.824568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c225d229a13f4c2395c359dc23b9a1c2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:31.832413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 949/1902] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:31.832413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b1ac8-6ba5-4951-88ef-f6f897165b1d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:31.836129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6e0b1ac8-6ba5-4951-88ef-f6f897165b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:31.845921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 954/1903] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:31 2022] DELETE /identity/v3/users/c225d229a13f4c2395c359dc23b9a1c2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:31.846497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:31.848145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:31.852641 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b1ac8-6ba5-4951-88ef-f6f897165b1d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:31.928078 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84d3eb-a340-4b02-8a2c-db347c78c3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:31.928983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84d3eb-a340-4b02-8a2c-db347c78c3dd admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.928983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84d3eb-a340-4b02-8a2c-db347c78c3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.929307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84d3eb-a340-4b02-8a2c-db347c78c3dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.929903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84d3eb-a340-4b02-8a2c-db347c78c3dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:31.932812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84d3eb-a340-4b02-8a2c-db347c78c3dd admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:31.936766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800472ca-15d1-465b-8a2b-a48781fcf7c2 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:31.942990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90313987-4381-4773-9ed5-f179af6707d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:31.942990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90313987-4381-4773-9ed5-f179af6707d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:31.942990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90313987-4381-4773-9ed5-f179af6707d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:31.944777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90313987-4381-4773-9ed5-f179af6707d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:31.945473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90313987-4381-4773-9ed5-f179af6707d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:31.948063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90313987-4381-4773-9ed5-f179af6707d5 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:31.987330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 950/1904] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:31 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:31.987330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84d3eb-a340-4b02-8a2c-db347c78c3dd admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:31.989634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-9c84d3eb-a340-4b02-8a2c-db347c78c3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:31.994018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 955/1905] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:31 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:31.994980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90313987-4381-4773-9ed5-f179af6707d5 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:32.031507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:32.084283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90313987-4381-4773-9ed5-f179af6707d5 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:32.093803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2a437-8c93-4cea-9f43-9c9816c4d784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:32.095093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2a437-8c93-4cea-9f43-9c9816c4d784 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.095603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2a437-8c93-4cea-9f43-9c9816c4d784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.096027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2a437-8c93-4cea-9f43-9c9816c4d784 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.099822 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2a437-8c93-4cea-9f43-9c9816c4d784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:32.102875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2a437-8c93-4cea-9f43-9c9816c4d784 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:32.117965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:32.135199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84d3eb-a340-4b02-8a2c-db347c78c3dd admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:32.140339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 956/1906] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:31 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:32.141627 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32406f-4fbc-4260-af04-2d8788c7150b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:32.142005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2a437-8c93-4cea-9f43-9c9816c4d784 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:32.142626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32406f-4fbc-4260-af04-2d8788c7150b admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.143067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32406f-4fbc-4260-af04-2d8788c7150b admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.143400 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32406f-4fbc-4260-af04-2d8788c7150b admin admin] PATH_INFO: `/v3/projects/6e07c5b4261240538a0a97b7e243cdd5/users/5ebb2efc6d9f4d8da4137c50203e7c6b/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.163573 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32406f-4fbc-4260-af04-2d8788c7150b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e07c5b4261240538a0a97b7e243cdd5, user_id=5ebb2efc6d9f4d8da4137c50203e7c6b, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:32.167730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32406f-4fbc-4260-af04-2d8788c7150b admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:32.173684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2a437-8c93-4cea-9f43-9c9816c4d784 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:32.191942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f226f-5857-41d8-ac71-a9a0b157947e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:32.192916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f226f-5857-41d8-ac71-a9a0b157947e admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.194117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f226f-5857-41d8-ac71-a9a0b157947e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.194493 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 951/1907] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:31 2022] PUT /identity/v3/projects/6e07c5b4261240538a0a97b7e243cdd5/users/5ebb2efc6d9f4d8da4137c50203e7c6b/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:32.194636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f226f-5857-41d8-ac71-a9a0b157947e admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.195420 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32406f-4fbc-4260-af04-2d8788c7150b admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:32.195567 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f226f-5857-41d8-ac71-a9a0b157947e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:32.201466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f226f-5857-41d8-ac71-a9a0b157947e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:32.250931 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 957/1908] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:32 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:32.252769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec1deb-619a-47e7-9c83-27751076fb2f None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.255076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec1deb-619a-47e7-9c83-27751076fb2f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.259021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec1deb-619a-47e7-9c83-27751076fb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.336472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-00ec1deb-619a-47e7-9c83-27751076fb2f None None] MFA Rules not processed for user `5ebb2efc6d9f4d8da4137c50203e7c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:32.339399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-00ec1deb-619a-47e7-9c83-27751076fb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:32.377579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32406f-4fbc-4260-af04-2d8788c7150b admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:32.384014 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a65e32-7c9d-4248-9af6-0d0d89ad9b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:32.385719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a65e32-7c9d-4248-9af6-0d0d89ad9b51 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.385719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a65e32-7c9d-4248-9af6-0d0d89ad9b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.386015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a65e32-7c9d-4248-9af6-0d0d89ad9b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.386740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a65e32-7c9d-4248-9af6-0d0d89ad9b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:32.388413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a65e32-7c9d-4248-9af6-0d0d89ad9b51 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:32.400430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 952/1909] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:32.401314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a65e32-7c9d-4248-9af6-0d0d89ad9b51 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:32.416365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a65e32-7c9d-4248-9af6-0d0d89ad9b51 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:32.422140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cf2b6-a350-47a1-8e0a-8be306247677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:32.422786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cf2b6-a350-47a1-8e0a-8be306247677 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.423067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cf2b6-a350-47a1-8e0a-8be306247677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.423411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cf2b6-a350-47a1-8e0a-8be306247677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.424006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cf2b6-a350-47a1-8e0a-8be306247677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:32.425726 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cf2b6-a350-47a1-8e0a-8be306247677 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:32.437339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 953/1910] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:32.438215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cf2b6-a350-47a1-8e0a-8be306247677 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:32.439824 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-106cf2b6-a350-47a1-8e0a-8be306247677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:32.452976 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cf2b6-a350-47a1-8e0a-8be306247677 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:32.454561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 958/1911] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:32.455461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec1deb-619a-47e7-9c83-27751076fb2f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:32.458159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:32.458654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.459005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.459383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] PATH_INFO: `/v3/projects/befeae50edc24147b1f79ec23333c483` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.460592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=befeae50edc24147b1f79ec23333c483)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:32.462718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:32.466525 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec1deb-619a-47e7-9c83-27751076fb2f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:32.471949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0559e7f-b4eb-43f1-9f77-a3005d9b055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:32.471949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0559e7f-b4eb-43f1-9f77-a3005d9b055f admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.472180 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0559e7f-b4eb-43f1-9f77-a3005d9b055f admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.472363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0559e7f-b4eb-43f1-9f77-a3005d9b055f admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.473031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0559e7f-b4eb-43f1-9f77-a3005d9b055f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:32.474675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0559e7f-b4eb-43f1-9f77-a3005d9b055f admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:32.486589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 959/1912] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:32.487476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0559e7f-b4eb-43f1-9f77-a3005d9b055f admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:32.498709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0559e7f-b4eb-43f1-9f77-a3005d9b055f admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:32.503802 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9995c-b3fe-4461-838b-96a6f5912915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:32.504424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9995c-b3fe-4461-838b-96a6f5912915 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.504748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9995c-b3fe-4461-838b-96a6f5912915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.505020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9995c-b3fe-4461-838b-96a6f5912915 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.505669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9995c-b3fe-4461-838b-96a6f5912915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:32.507338 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9995c-b3fe-4461-838b-96a6f5912915 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:32.517552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'befeae50edc24147b1f79ec23333c483', 'request_id': 'req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:32.519586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 960/1913] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:32.520417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9995c-b3fe-4461-838b-96a6f5912915 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:32.529781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'befeae50edc24147b1f79ec23333c483', 'request_id': 'req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:32.533937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9995c-b3fe-4461-838b-96a6f5912915 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:32.536400 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'befeae50edc24147b1f79ec23333c483', 'request_id': 'req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:32.549839 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] The token cache is being invalidate because project befeae50edc24147b1f79ec23333c483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:32.550130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:32.550830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:32.552752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 954/1914] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:32 2022] DELETE /identity/v3/projects/befeae50edc24147b1f79ec23333c483 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:32.553443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:32.555221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:32.569382 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:32.594719 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb5dd8-a80b-4f14-8d3a-df30cb07e9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:32.595334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb5dd8-a80b-4f14-8d3a-df30cb07e9c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.595627 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb5dd8-a80b-4f14-8d3a-df30cb07e9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.595895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb5dd8-a80b-4f14-8d3a-df30cb07e9c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.596538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb5dd8-a80b-4f14-8d3a-df30cb07e9c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:32.598508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb5dd8-a80b-4f14-8d3a-df30cb07e9c7 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:32.632226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 961/1915] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:32 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:32.633553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb5dd8-a80b-4f14-8d3a-df30cb07e9c7 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:32.635875 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c273c7-0e5d-4ea4-b435-62644e8bcbd9 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:32.636900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7feb5dd8-a80b-4f14-8d3a-df30cb07e9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:32.679145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:32.719667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb5dd8-a80b-4f14-8d3a-df30cb07e9c7 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:32.725305 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1bd13-b8d8-4040-b543-dd0ac443b860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:32.726109 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1bd13-b8d8-4040-b543-dd0ac443b860 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.726494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1bd13-b8d8-4040-b543-dd0ac443b860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.726776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1bd13-b8d8-4040-b543-dd0ac443b860 admin admin] PATH_INFO: `/v3/projects/8019caca03bd4cd7b440c3b15e5e2f6f/users/1a40559b69e141cb94d0420e4a8405be/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.736244 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95b148-a99c-4029-9999-827f5c3df98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:32.737047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95b148-a99c-4029-9999-827f5c3df98d admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.737860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95b148-a99c-4029-9999-827f5c3df98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.738214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95b148-a99c-4029-9999-827f5c3df98d admin admin] PATH_INFO: `/v3/projects/92296969740f43bf97f12b77fa4ebde1/users/3835a910d73146d5933b2b72e6d02ba7/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.746251 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1bd13-b8d8-4040-b543-dd0ac443b860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8019caca03bd4cd7b440c3b15e5e2f6f, user_id=1a40559b69e141cb94d0420e4a8405be, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:32.749272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1bd13-b8d8-4040-b543-dd0ac443b860 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:32.758331 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95b148-a99c-4029-9999-827f5c3df98d admin admin] RBAC: Authorizing `identity:create_grant(project_id=92296969740f43bf97f12b77fa4ebde1, user_id=3835a910d73146d5933b2b72e6d02ba7, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:32.761065 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95b148-a99c-4029-9999-827f5c3df98d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:32.773226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 962/1916] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:32 2022] PUT /identity/v3/projects/8019caca03bd4cd7b440c3b15e5e2f6f/users/1a40559b69e141cb94d0420e4a8405be/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:32.774513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1eed1-bc1f-4e3d-9653-f11f292ea2ee None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.776148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1eed1-bc1f-4e3d-9653-f11f292ea2ee None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.778040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1eed1-bc1f-4e3d-9653-f11f292ea2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.781979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 955/1917] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:32 2022] PUT /identity/v3/projects/92296969740f43bf97f12b77fa4ebde1/users/3835a910d73146d5933b2b72e6d02ba7/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:32.788580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95b148-a99c-4029-9999-827f5c3df98d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:32.790960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0b95b148-a99c-4029-9999-827f5c3df98d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:32.801906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-1ea1eed1-bc1f-4e3d-9653-f11f292ea2ee None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:32.804053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1ea1eed1-bc1f-4e3d-9653-f11f292ea2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:32.876667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95b148-a99c-4029-9999-827f5c3df98d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:32.891148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a127ff-26c7-4c6e-ba07-97f003e86087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:32.891148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a127ff-26c7-4c6e-ba07-97f003e86087 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.891148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a127ff-26c7-4c6e-ba07-97f003e86087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.891148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a127ff-26c7-4c6e-ba07-97f003e86087 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.891699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a127ff-26c7-4c6e-ba07-97f003e86087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:32.891774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 963/1918] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:32.891854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a127ff-26c7-4c6e-ba07-97f003e86087 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:32.891930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343abb3-6979-40df-8ac2-d836baac5fe2 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.891930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343abb3-6979-40df-8ac2-d836baac5fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.891930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343abb3-6979-40df-8ac2-d836baac5fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.918186 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 956/1919] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:32 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:32.919783 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23377581-555c-4d42-b083-828dcce5b88a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:32.920281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23377581-555c-4d42-b083-828dcce5b88a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:32.920568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23377581-555c-4d42-b083-828dcce5b88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:32.931449 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b343abb3-6979-40df-8ac2-d836baac5fe2 None None] MFA Rules not processed for user `1a40559b69e141cb94d0420e4a8405be`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:32.933546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b343abb3-6979-40df-8ac2-d836baac5fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:32.966874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-23377581-555c-4d42-b083-828dcce5b88a None None] MFA Rules not processed for user `3835a910d73146d5933b2b72e6d02ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:32.968163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-23377581-555c-4d42-b083-828dcce5b88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:32.989487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:33.046745 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 964/1920] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:33.052777 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb26ad-a280-4a9e-9d55-0e3fb046bec3 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.053218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb26ad-a280-4a9e-9d55-0e3fb046bec3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.053755 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 957/1921] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:33.053868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb26ad-a280-4a9e-9d55-0e3fb046bec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.054379 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23377581-555c-4d42-b083-828dcce5b88a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:33.079982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23377581-555c-4d42-b083-828dcce5b88a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:33.086292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af8d65-15f6-4dd9-a368-428a70767dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:33.087233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-53bb26ad-a280-4a9e-9d55-0e3fb046bec3 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:33.087504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af8d65-15f6-4dd9-a368-428a70767dcf admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.087974 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af8d65-15f6-4dd9-a368-428a70767dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.088419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af8d65-15f6-4dd9-a368-428a70767dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.088586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-53bb26ad-a280-4a9e-9d55-0e3fb046bec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:33.089500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af8d65-15f6-4dd9-a368-428a70767dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:33.091332 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af8d65-15f6-4dd9-a368-428a70767dcf admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:33.097754 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 965/1922] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:33.098776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb26ad-a280-4a9e-9d55-0e3fb046bec3 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:33.109571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 958/1923] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:33.111384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb26ad-a280-4a9e-9d55-0e3fb046bec3 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:33.111605 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af8d65-15f6-4dd9-a368-428a70767dcf admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:33.115645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c3af8d65-15f6-4dd9-a368-428a70767dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:33.119518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f09782-c999-4d02-9c13-467269f36740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:33.120091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f09782-c999-4d02-9c13-467269f36740 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.121372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f09782-c999-4d02-9c13-467269f36740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.121372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f09782-c999-4d02-9c13-467269f36740 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.121372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f09782-c999-4d02-9c13-467269f36740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:33.123291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f09782-c999-4d02-9c13-467269f36740 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:33.172701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 966/1924] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:33 2022] POST /identity/v3/projects => generated 365 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:33.174096 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f09782-c999-4d02-9c13-467269f36740 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:33.192749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af8d65-15f6-4dd9-a368-428a70767dcf admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:33.198875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:33.266790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f09782-c999-4d02-9c13-467269f36740 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:33.268571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282b48a-b2c1-4178-ba5a-12f4ec5430b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:33.269413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282b48a-b2c1-4178-ba5a-12f4ec5430b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.269688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282b48a-b2c1-4178-ba5a-12f4ec5430b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.269954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282b48a-b2c1-4178-ba5a-12f4ec5430b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.273227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec6e31-d721-481f-844d-84390c6ae965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:33.273481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec6e31-d721-481f-844d-84390c6ae965 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.273800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec6e31-d721-481f-844d-84390c6ae965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.273950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec6e31-d721-481f-844d-84390c6ae965 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.274575 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec6e31-d721-481f-844d-84390c6ae965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:17:33.274886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec6e31-d721-481f-844d-84390c6ae965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:33.276590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec6e31-d721-481f-844d-84390c6ae965 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:33.294643 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 967/1925] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:17:33.299213 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec6e31-d721-481f-844d-84390c6ae965 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:33.312280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec6e31-d721-481f-844d-84390c6ae965 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:33.317975 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff9db7-5622-4608-bf46-d2c52ad80eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:33.318676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff9db7-5622-4608-bf46-d2c52ad80eba admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.318962 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff9db7-5622-4608-bf46-d2c52ad80eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.319325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff9db7-5622-4608-bf46-d2c52ad80eba admin admin] PATH_INFO: `/v3/projects/75d36fe7dc5b41b5a281cbcab40743a9/users/6bb0e1913b004a698f2061051c454cee/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.345457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff9db7-5622-4608-bf46-d2c52ad80eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=75d36fe7dc5b41b5a281cbcab40743a9, user_id=6bb0e1913b004a698f2061051c454cee, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:33.349034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff9db7-5622-4608-bf46-d2c52ad80eba admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:33.358471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282b48a-b2c1-4178-ba5a-12f4ec5430b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:33.362292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282b48a-b2c1-4178-ba5a-12f4ec5430b3 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:33.375796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 968/1926] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:33 2022] PUT /identity/v3/projects/75d36fe7dc5b41b5a281cbcab40743a9/users/6bb0e1913b004a698f2061051c454cee/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:33.376752 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff9db7-5622-4608-bf46-d2c52ad80eba admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:33.411784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 959/1927] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:33.414189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282b48a-b2c1-4178-ba5a-12f4ec5430b3 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:33.432686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282b48a-b2c1-4178-ba5a-12f4ec5430b3 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:33.464966 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:33.470301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff9db7-5622-4608-bf46-d2c52ad80eba admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:33.476288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf518e0-bda4-446f-a088-7741c026ee1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:33.476931 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf518e0-bda4-446f-a088-7741c026ee1f admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.477217 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf518e0-bda4-446f-a088-7741c026ee1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.478233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf518e0-bda4-446f-a088-7741c026ee1f admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.478912 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf518e0-bda4-446f-a088-7741c026ee1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:33.481609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf518e0-bda4-446f-a088-7741c026ee1f admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:33.511783 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 969/1928] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:33 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:33.513174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf518e0-bda4-446f-a088-7741c026ee1f admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:33.526511 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf518e0-bda4-446f-a088-7741c026ee1f admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:33.530214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13862d4e-d09f-4fe8-87a5-cf8eff6b3d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:33.530950 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13862d4e-d09f-4fe8-87a5-cf8eff6b3d7d admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.531476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13862d4e-d09f-4fe8-87a5-cf8eff6b3d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.531937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13862d4e-d09f-4fe8-87a5-cf8eff6b3d7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.532566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a5788-a715-430c-917c-8ddb32bfa305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:33.532724 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13862d4e-d09f-4fe8-87a5-cf8eff6b3d7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:33.533610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a5788-a715-430c-917c-8ddb32bfa305 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.534033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a5788-a715-430c-917c-8ddb32bfa305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.534429 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a5788-a715-430c-917c-8ddb32bfa305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.534891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13862d4e-d09f-4fe8-87a5-cf8eff6b3d7d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:33.535384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a5788-a715-430c-917c-8ddb32bfa305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:33.537105 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a5788-a715-430c-917c-8ddb32bfa305 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:33.550859 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 970/1929] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:33.555237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a5788-a715-430c-917c-8ddb32bfa305 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:33.568434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a5788-a715-430c-917c-8ddb32bfa305 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:33.570183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 960/1930] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:33 2022] POST /identity/v3/projects => generated 379 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:33.571340 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13862d4e-d09f-4fe8-87a5-cf8eff6b3d7d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:33.656337 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13862d4e-d09f-4fe8-87a5-cf8eff6b3d7d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:33.661683 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a305c-890d-4d85-8727-b01dcde5d6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:33.662578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a305c-890d-4d85-8727-b01dcde5d6f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.662937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a305c-890d-4d85-8727-b01dcde5d6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.663297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a305c-890d-4d85-8727-b01dcde5d6f3 admin admin] PATH_INFO: `/v3/projects/75d36fe7dc5b41b5a281cbcab40743a9/users/6bb0e1913b004a698f2061051c454cee/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.666083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a305c-890d-4d85-8727-b01dcde5d6f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75d36fe7dc5b41b5a281cbcab40743a9, user_id=6bb0e1913b004a698f2061051c454cee, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:33.670015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a305c-890d-4d85-8727-b01dcde5d6f3 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:33.677100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700532d5-4c8f-4998-9aeb-0b55a873d2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:33.677874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700532d5-4c8f-4998-9aeb-0b55a873d2ad admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.678327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700532d5-4c8f-4998-9aeb-0b55a873d2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.678677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700532d5-4c8f-4998-9aeb-0b55a873d2ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.679516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700532d5-4c8f-4998-9aeb-0b55a873d2ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:33.681281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700532d5-4c8f-4998-9aeb-0b55a873d2ad admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:33.684454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 961/1931] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:33 2022] PUT /identity/v3/projects/75d36fe7dc5b41b5a281cbcab40743a9/users/6bb0e1913b004a698f2061051c454cee/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:33.686444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a305c-890d-4d85-8727-b01dcde5d6f3 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:33.710463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:33.722886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 971/1932] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:33.724896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb6df2-1717-4ff1-aec1-6c07623436d3 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.725392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb6df2-1717-4ff1-aec1-6c07623436d3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.725966 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb6df2-1717-4ff1-aec1-6c07623436d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.764450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f8eb6df2-1717-4ff1-aec1-6c07623436d3 None None] MFA Rules not processed for user `6bb0e1913b004a698f2061051c454cee`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:33.766109 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f8eb6df2-1717-4ff1-aec1-6c07623436d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:33.796540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a305c-890d-4d85-8727-b01dcde5d6f3 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:33.804899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242f2df-8a71-4d68-9798-4a2fabd2ec12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:33.805747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242f2df-8a71-4d68-9798-4a2fabd2ec12 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.806209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242f2df-8a71-4d68-9798-4a2fabd2ec12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.806332 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242f2df-8a71-4d68-9798-4a2fabd2ec12 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.808028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242f2df-8a71-4d68-9798-4a2fabd2ec12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:33.810147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242f2df-8a71-4d68-9798-4a2fabd2ec12 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:33.844595 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 962/1933] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:33 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:33.845435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242f2df-8a71-4d68-9798-4a2fabd2ec12 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:33.864349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242f2df-8a71-4d68-9798-4a2fabd2ec12 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:33.875891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec93587-3fa9-4d06-8f13-e1dfb224feb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:33.877189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec93587-3fa9-4d06-8f13-e1dfb224feb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:33.877535 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec93587-3fa9-4d06-8f13-e1dfb224feb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:33.877849 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec93587-3fa9-4d06-8f13-e1dfb224feb9 admin admin] PATH_INFO: `/v3/projects/f092c4f68a874b3dae48d921830c8cfc/users/ecd163d5d3944a168f5c063be0e592c4/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:33.894010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 972/1934] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:33.896701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb6df2-1717-4ff1-aec1-6c07623436d3 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:33.923878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec93587-3fa9-4d06-8f13-e1dfb224feb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f092c4f68a874b3dae48d921830c8cfc, user_id=ecd163d5d3944a168f5c063be0e592c4, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:33.931719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec93587-3fa9-4d06-8f13-e1dfb224feb9 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:33.957479 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb6df2-1717-4ff1-aec1-6c07623436d3 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:33.974112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 963/1935] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:33 2022] PUT /identity/v3/projects/f092c4f68a874b3dae48d921830c8cfc/users/ecd163d5d3944a168f5c063be0e592c4/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:33.977917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec93587-3fa9-4d06-8f13-e1dfb224feb9 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:34.000975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:34.077248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c294de2-74ea-4374-8054-65342ea8ffdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:34.078172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c294de2-74ea-4374-8054-65342ea8ffdd admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:34.078991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c294de2-74ea-4374-8054-65342ea8ffdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:34.080109 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c294de2-74ea-4374-8054-65342ea8ffdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:34.080109 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c294de2-74ea-4374-8054-65342ea8ffdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:34.082868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c294de2-74ea-4374-8054-65342ea8ffdd admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:34.100117 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec93587-3fa9-4d06-8f13-e1dfb224feb9 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:34.100815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 973/1936] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:34.101811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c294de2-74ea-4374-8054-65342ea8ffdd admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:34.106138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fb0d6-b182-47e5-889e-6d16ab89006d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:34.106789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fb0d6-b182-47e5-889e-6d16ab89006d service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:34.107874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fb0d6-b182-47e5-889e-6d16ab89006d service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:34.107874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fb0d6-b182-47e5-889e-6d16ab89006d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:34.113508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c294de2-74ea-4374-8054-65342ea8ffdd admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:34.120030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be56966-af33-419c-bb1a-4cc0f7059d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:34.120676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be56966-af33-419c-bb1a-4cc0f7059d93 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:34.120960 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be56966-af33-419c-bb1a-4cc0f7059d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:34.121242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be56966-af33-419c-bb1a-4cc0f7059d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:34.121901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be56966-af33-419c-bb1a-4cc0f7059d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:34.123816 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be56966-af33-419c-bb1a-4cc0f7059d93 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:34.135654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 974/1937] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:34.136506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be56966-af33-419c-bb1a-4cc0f7059d93 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:34.151263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be56966-af33-419c-bb1a-4cc0f7059d93 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:34.157011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46cb67-923b-4477-943a-d56767949714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:34.157615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46cb67-923b-4477-943a-d56767949714 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:34.157899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46cb67-923b-4477-943a-d56767949714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:34.158177 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46cb67-923b-4477-943a-d56767949714 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:34.158866 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46cb67-923b-4477-943a-d56767949714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:34.160508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46cb67-923b-4477-943a-d56767949714 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:34.174494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 975/1938] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:34.178988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46cb67-923b-4477-943a-d56767949714 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:34.190407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46cb67-923b-4477-943a-d56767949714 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:34.195759 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fb0d6-b182-47e5-889e-6d16ab89006d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:34.197680 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fb0d6-b182-47e5-889e-6d16ab89006d service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:34.204030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87f170-0b60-41f2-978f-5ae1cc0a8f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:34.205132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87f170-0b60-41f2-978f-5ae1cc0a8f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:34.205589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87f170-0b60-41f2-978f-5ae1cc0a8f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:34.205943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87f170-0b60-41f2-978f-5ae1cc0a8f02 admin admin] PATH_INFO: `/v3/projects/f092c4f68a874b3dae48d921830c8cfc/users/ecd163d5d3944a168f5c063be0e592c4/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:34.208633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87f170-0b60-41f2-978f-5ae1cc0a8f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f092c4f68a874b3dae48d921830c8cfc, user_id=ecd163d5d3944a168f5c063be0e592c4, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:34.209063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 964/1939] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:34.209972 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fb0d6-b182-47e5-889e-6d16ab89006d service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:34.213048 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87f170-0b60-41f2-978f-5ae1cc0a8f02 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:34.222494 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fb0d6-b182-47e5-889e-6d16ab89006d service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:34.235208 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 976/1940] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:34 2022] PUT /identity/v3/projects/f092c4f68a874b3dae48d921830c8cfc/users/ecd163d5d3944a168f5c063be0e592c4/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:34.246394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfe985-ca3d-40f2-a620-08e04116f5fd None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:34.246613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfe985-ca3d-40f2-a620-08e04116f5fd None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:34.247061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfe985-ca3d-40f2-a620-08e04116f5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:34.282636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-e1cfe985-ca3d-40f2-a620-08e04116f5fd None None] MFA Rules not processed for user `ecd163d5d3944a168f5c063be0e592c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:34.284237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e1cfe985-ca3d-40f2-a620-08e04116f5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:34.323025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:34.359176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3b267-412a-4702-a6ce-2fb0a4b9b47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:34.360422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b267-412a-4702-a6ce-2fb0a4b9b47a admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:34.360712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b267-412a-4702-a6ce-2fb0a4b9b47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:34.361004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b267-412a-4702-a6ce-2fb0a4b9b47a admin admin] PATH_INFO: `/v3/projects/f4754913b2af4b04a2b740a251c33092/users/19fcbd1f8f3a4dc98f5ded776563b5f3/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:34.382189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3b267-412a-4702-a6ce-2fb0a4b9b47a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4754913b2af4b04a2b740a251c33092, user_id=19fcbd1f8f3a4dc98f5ded776563b5f3, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:34.387960 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 977/1941] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:34.388479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3b267-412a-4702-a6ce-2fb0a4b9b47a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:34.396158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfe985-ca3d-40f2-a620-08e04116f5fd None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:34.405926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 965/1942] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:34 2022] PUT /identity/v3/projects/f4754913b2af4b04a2b740a251c33092/users/19fcbd1f8f3a4dc98f5ded776563b5f3/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:34.420644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df39b5-74fd-455e-9748-88859a9c65fc None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:34.420644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df39b5-74fd-455e-9748-88859a9c65fc None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:34.420644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df39b5-74fd-455e-9748-88859a9c65fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:34.465811 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-32df39b5-74fd-455e-9748-88859a9c65fc None None] MFA Rules not processed for user `19fcbd1f8f3a4dc98f5ded776563b5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:34.467039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-32df39b5-74fd-455e-9748-88859a9c65fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:34.496926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfe985-ca3d-40f2-a620-08e04116f5fd None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:34.501805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:34.558377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 966/1943] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:34.562817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df39b5-74fd-455e-9748-88859a9c65fc None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:34.601506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2965ec1-b316-415d-8fb9-364039a13db9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:34.602349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2965ec1-b316-415d-8fb9-364039a13db9 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:34.602695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2965ec1-b316-415d-8fb9-364039a13db9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:34.603860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2965ec1-b316-415d-8fb9-364039a13db9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:34.643774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df39b5-74fd-455e-9748-88859a9c65fc None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:34.659140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc1942-5761-47d1-8f34-1a1760e72b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:34.659140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc1942-5761-47d1-8f34-1a1760e72b6f admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:34.659140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc1942-5761-47d1-8f34-1a1760e72b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:34.659140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc1942-5761-47d1-8f34-1a1760e72b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:34.659581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc1942-5761-47d1-8f34-1a1760e72b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:34.659581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc1942-5761-47d1-8f34-1a1760e72b6f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:34.702464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 967/1944] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:34 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:17:34.703464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc1942-5761-47d1-8f34-1a1760e72b6f admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:34.704230 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2965ec1-b316-415d-8fb9-364039a13db9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:34.706171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2965ec1-b316-415d-8fb9-364039a13db9 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:34.722626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 978/1945] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:17:34.814337 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc1942-5761-47d1-8f34-1a1760e72b6f admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:34.822941 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62b9ec-88b4-4e72-aed8-f522e4c595ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:34.823748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62b9ec-88b4-4e72-aed8-f522e4c595ac admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:34.824282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62b9ec-88b4-4e72-aed8-f522e4c595ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:34.824564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62b9ec-88b4-4e72-aed8-f522e4c595ac admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:34.826041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62b9ec-88b4-4e72-aed8-f522e4c595ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:34.828193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62b9ec-88b4-4e72-aed8-f522e4c595ac admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:34.859742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2965ec1-b316-415d-8fb9-364039a13db9 service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:34.868630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 968/1946] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:34 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:34.869624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62b9ec-88b4-4e72-aed8-f522e4c595ac admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:34.870890 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2965ec1-b316-415d-8fb9-364039a13db9 service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:34.880413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62b9ec-88b4-4e72-aed8-f522e4c595ac admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:34.888853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:34.889364 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:34.889637 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:34.889909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] PATH_INFO: `/v3/users/5ebb2efc6d9f4d8da4137c50203e7c6b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:34.891057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ebb2efc6d9f4d8da4137c50203e7c6b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:34.893292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:34.915475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:34.937034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e47c78-4db2-4ef3-a499-cc60ac077c6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:34.938104 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e47c78-4db2-4ef3-a499-cc60ac077c6f service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:34.939216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e47c78-4db2-4ef3-a499-cc60ac077c6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:34.939780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e47c78-4db2-4ef3-a499-cc60ac077c6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:34.941719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:34.955431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ebb2efc6d9f4d8da4137c50203e7c6b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:34.969276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ebb2efc6d9f4d8da4137c50203e7c6b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:34.970380 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ebb2efc6d9f4d8da4137c50203e7c6b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:34.971349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ebb2efc6d9f4d8da4137c50203e7c6b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:35.004146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ebb2efc6d9f4d8da4137c50203e7c6b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:35.042714 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 969/1947] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:34 2022] DELETE /identity/v3/users/5ebb2efc6d9f4d8da4137c50203e7c6b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:35.043520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:35.046967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e47c78-4db2-4ef3-a499-cc60ac077c6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:35.047758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:35.049815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e47c78-4db2-4ef3-a499-cc60ac077c6f service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:35.065569 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 979/1948] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:35.066925 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab731-fc1a-4602-8e05-6881bae2e772 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:35.067246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab731-fc1a-4602-8e05-6881bae2e772 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:35.067521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab731-fc1a-4602-8e05-6881bae2e772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:35.098286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-943ab731-fc1a-4602-8e05-6881bae2e772 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:35.099581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-943ab731-fc1a-4602-8e05-6881bae2e772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:35.224126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 980/1949] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:35.233851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a56ef-9d84-4512-a8f0-1d7c1361ac21 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:35.236969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ab731-fc1a-4602-8e05-6881bae2e772 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:35.238688 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-943ab731-fc1a-4602-8e05-6881bae2e772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:35.252576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1d5f1-7f51-4b21-ba09-67ac23986d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:35.253329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1d5f1-7f51-4b21-ba09-67ac23986d02 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:35.253605 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1d5f1-7f51-4b21-ba09-67ac23986d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:35.253866 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1d5f1-7f51-4b21-ba09-67ac23986d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:35.254524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1d5f1-7f51-4b21-ba09-67ac23986d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:35.256305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1d5f1-7f51-4b21-ba09-67ac23986d02 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:35.284553 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 970/1950] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:35.292994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1d5f1-7f51-4b21-ba09-67ac23986d02 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:35.305655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1d5f1-7f51-4b21-ba09-67ac23986d02 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:35.339409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fb864-6f21-4969-b9a8-ca35455c3351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:35.340049 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fb864-6f21-4969-b9a8-ca35455c3351 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:35.340373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fb864-6f21-4969-b9a8-ca35455c3351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:35.340639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fb864-6f21-4969-b9a8-ca35455c3351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:35.341308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fb864-6f21-4969-b9a8-ca35455c3351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:35.343274 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fb864-6f21-4969-b9a8-ca35455c3351 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:35.364538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ab731-fc1a-4602-8e05-6881bae2e772 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:35.369330 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e3c2-80e2-46c7-a4c7-af8b8a65505c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:35.370188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e3c2-80e2-46c7-a4c7-af8b8a65505c service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:35.370645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e3c2-80e2-46c7-a4c7-af8b8a65505c service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:35.371086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e3c2-80e2-46c7-a4c7-af8b8a65505c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:35.371227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 971/1951] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:35.372342 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fb864-6f21-4969-b9a8-ca35455c3351 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:35.382896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2e3c2-80e2-46c7-a4c7-af8b8a65505c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:35.385313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2e3c2-80e2-46c7-a4c7-af8b8a65505c service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:35.400767 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 981/1952] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:35.413103 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fb864-6f21-4969-b9a8-ca35455c3351 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:35.419927 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178bb5b-a64d-4fa5-844c-9debd48a1d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:35.420620 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178bb5b-a64d-4fa5-844c-9debd48a1d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:35.420949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178bb5b-a64d-4fa5-844c-9debd48a1d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:35.421507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178bb5b-a64d-4fa5-844c-9debd48a1d82 admin admin] PATH_INFO: `/v3/projects/11fb23877ddd45f9981a8cd01907323c/users/d1c4cb37918e4945a0b2812b381993c8/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:35.463905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178bb5b-a64d-4fa5-844c-9debd48a1d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fb23877ddd45f9981a8cd01907323c, user_id=d1c4cb37918e4945a0b2812b381993c8, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:35.468473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178bb5b-a64d-4fa5-844c-9debd48a1d82 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:35.492453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 972/1953] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:35 2022] PUT /identity/v3/projects/11fb23877ddd45f9981a8cd01907323c/users/d1c4cb37918e4945a0b2812b381993c8/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:35.507579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2e79b-9c01-42be-abfb-54fd6412df47 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:35.508055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2e79b-9c01-42be-abfb-54fd6412df47 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:35.508597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2e79b-9c01-42be-abfb-54fd6412df47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:35.550009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a0a2e79b-9c01-42be-abfb-54fd6412df47 None None] MFA Rules not processed for user `d1c4cb37918e4945a0b2812b381993c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:35.551808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a0a2e79b-9c01-42be-abfb-54fd6412df47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:35.602386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:35.661194 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 982/1954] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:35.665723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178bb5b-a64d-4fa5-844c-9debd48a1d82 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:35.749700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178bb5b-a64d-4fa5-844c-9debd48a1d82 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:35.757400 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f49218-921a-4492-85f1-550df4bfce47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:35.758033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f49218-921a-4492-85f1-550df4bfce47 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:35.758307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f49218-921a-4492-85f1-550df4bfce47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:35.758575 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f49218-921a-4492-85f1-550df4bfce47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:35.759900 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f49218-921a-4492-85f1-550df4bfce47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:35.761854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f49218-921a-4492-85f1-550df4bfce47 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:35.786702 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2e79b-9c01-42be-abfb-54fd6412df47 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:35.789565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a0a2e79b-9c01-42be-abfb-54fd6412df47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:35.792990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:35.808502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 973/1955] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:35 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:35.810027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f49218-921a-4492-85f1-550df4bfce47 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:35.831597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2e79b-9c01-42be-abfb-54fd6412df47 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:35.901225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f49218-921a-4492-85f1-550df4bfce47 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:35.929364 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a5db9-1973-4a7d-aacd-490dba0a460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:35.930263 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a5db9-1973-4a7d-aacd-490dba0a460f admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:35.930780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a5db9-1973-4a7d-aacd-490dba0a460f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:35.931409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a5db9-1973-4a7d-aacd-490dba0a460f admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:35.932644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a5db9-1973-4a7d-aacd-490dba0a460f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:35.935685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a5db9-1973-4a7d-aacd-490dba0a460f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:35.945647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:35.946341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:35.946623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:35.946889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] PATH_INFO: `/v3/projects/6e07c5b4261240538a0a97b7e243cdd5` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:35.949699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e07c5b4261240538a0a97b7e243cdd5)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:35.956607 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:35.972526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 974/1956] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:35 2022] POST /identity/v3/users => generated 295 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:17:35.976901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a5db9-1973-4a7d-aacd-490dba0a460f admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:36.000731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a5db9-1973-4a7d-aacd-490dba0a460f admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:36.026570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca833d-6a6e-45da-9523-0849b1022121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:36.027711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca833d-6a6e-45da-9523-0849b1022121 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:36.028307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca833d-6a6e-45da-9523-0849b1022121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:36.028859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca833d-6a6e-45da-9523-0849b1022121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:36.029838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca833d-6a6e-45da-9523-0849b1022121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:36.031933 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca833d-6a6e-45da-9523-0849b1022121 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:36.048248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 975/1957] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:36.052218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca833d-6a6e-45da-9523-0849b1022121 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:36.060135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e07c5b4261240538a0a97b7e243cdd5', 'request_id': 'req-46e4f9b5-1187-46af-b9da-de7eeafd23bf'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:36.069328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca833d-6a6e-45da-9523-0849b1022121 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:36.089888 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e07c5b4261240538a0a97b7e243cdd5', 'request_id': 'req-46e4f9b5-1187-46af-b9da-de7eeafd23bf'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:36.101481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e07c5b4261240538a0a97b7e243cdd5', 'request_id': 'req-46e4f9b5-1187-46af-b9da-de7eeafd23bf'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:36.130537 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] The token cache is being invalidate because project 6e07c5b4261240538a0a97b7e243cdd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:36.132981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:36.135833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:36.138116 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 983/1958] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:35 2022] DELETE /identity/v3/projects/6e07c5b4261240538a0a97b7e243cdd5 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:36.142407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:36.146331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:36.160812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59813450-9265-49bb-ad1e-c9888f619edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:36.162053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59813450-9265-49bb-ad1e-c9888f619edf admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:36.162440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59813450-9265-49bb-ad1e-c9888f619edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:36.162819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59813450-9265-49bb-ad1e-c9888f619edf admin admin] PATH_INFO: `/v3/projects/9b6810fa0c024f4e9002a7f519d2c111/users/5c496a451d7b4bb19fec448cb6d5ba59/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:36.183738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59813450-9265-49bb-ad1e-c9888f619edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6810fa0c024f4e9002a7f519d2c111, user_id=5c496a451d7b4bb19fec448cb6d5ba59, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:36.189015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59813450-9265-49bb-ad1e-c9888f619edf admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:36.210810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 976/1959] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:36 2022] PUT /identity/v3/projects/9b6810fa0c024f4e9002a7f519d2c111/users/5c496a451d7b4bb19fec448cb6d5ba59/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:36.216760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59813450-9265-49bb-ad1e-c9888f619edf admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:36.219489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-59813450-9265-49bb-ad1e-c9888f619edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:36.266392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:36.275216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4f9b5-1187-46af-b9da-de7eeafd23bf admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:36.319355 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59813450-9265-49bb-ad1e-c9888f619edf admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:36.327599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84329eba-3f69-4b05-b806-cae9d737de4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:36.328569 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84329eba-3f69-4b05-b806-cae9d737de4f admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:36.329002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84329eba-3f69-4b05-b806-cae9d737de4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:36.329569 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84329eba-3f69-4b05-b806-cae9d737de4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:36.330362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84329eba-3f69-4b05-b806-cae9d737de4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:36.332500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84329eba-3f69-4b05-b806-cae9d737de4f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:36.335662 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:36.354801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 977/1960] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:36.356768 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84329eba-3f69-4b05-b806-cae9d737de4f admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:36.374852 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84329eba-3f69-4b05-b806-cae9d737de4f admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:36.382233 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7770e-9b6c-4c8e-b2ce-fcccfe9ce90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:36.383475 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7770e-9b6c-4c8e-b2ce-fcccfe9ce90e admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:36.383691 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:36.384369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7770e-9b6c-4c8e-b2ce-fcccfe9ce90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:36.385742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:36.385856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:36.386636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] PATH_INFO: `/v3/users/6bb0e1913b004a698f2061051c454cee` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:36.387630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7770e-9b6c-4c8e-b2ce-fcccfe9ce90e admin admin] PATH_INFO: `/v3/projects/9b6810fa0c024f4e9002a7f519d2c111/users/5c496a451d7b4bb19fec448cb6d5ba59/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:36.392599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7770e-9b6c-4c8e-b2ce-fcccfe9ce90e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6810fa0c024f4e9002a7f519d2c111, user_id=5c496a451d7b4bb19fec448cb6d5ba59, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:36.393741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb0e1913b004a698f2061051c454cee)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:36.396760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7770e-9b6c-4c8e-b2ce-fcccfe9ce90e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:36.403234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:36.443692 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 978/1961] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:36 2022] PUT /identity/v3/projects/9b6810fa0c024f4e9002a7f519d2c111/users/5c496a451d7b4bb19fec448cb6d5ba59/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:36.464998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d4f3e-6851-48e3-be0c-e0da66c90992 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:36.465297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d4f3e-6851-48e3-be0c-e0da66c90992 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:36.465588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d4f3e-6851-48e3-be0c-e0da66c90992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:36.515742 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-6b0d4f3e-6851-48e3-be0c-e0da66c90992 None None] MFA Rules not processed for user `5c496a451d7b4bb19fec448cb6d5ba59`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:36.516911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6b0d4f3e-6851-48e3-be0c-e0da66c90992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:36.518865 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb0e1913b004a698f2061051c454cee'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:36.552212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb0e1913b004a698f2061051c454cee'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:36.565249 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb0e1913b004a698f2061051c454cee'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:36.566327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb0e1913b004a698f2061051c454cee'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:36.568557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb0e1913b004a698f2061051c454cee'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:36.603474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 984/1962] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:36 2022] DELETE /identity/v3/users/6bb0e1913b004a698f2061051c454cee => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:36.632685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 979/1963] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:36.645886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:36.647832 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:36.753593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd918f-8171-478b-ba0e-2f0f40a65455 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:36.759348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7cfb9-909c-424f-920e-d8bbde9ad308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:36.761061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7cfb9-909c-424f-920e-d8bbde9ad308 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:36.761663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7cfb9-909c-424f-920e-d8bbde9ad308 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:36.761949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7cfb9-909c-424f-920e-d8bbde9ad308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:36.815155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:36.875418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7cfb9-909c-424f-920e-d8bbde9ad308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:36.878234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7cfb9-909c-424f-920e-d8bbde9ad308 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:36.895691 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 985/1964] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:17:36.981303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d4f3e-6851-48e3-be0c-e0da66c90992 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:37.154029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7cfb9-909c-424f-920e-d8bbde9ad308 service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:37.155466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-91e7cfb9-909c-424f-920e-d8bbde9ad308 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:37.168110 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d4f3e-6851-48e3-be0c-e0da66c90992 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:37.172746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f2796-674b-4512-9258-e13745889543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:37.173386 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f2796-674b-4512-9258-e13745889543 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:37.173668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f2796-674b-4512-9258-e13745889543 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:37.173937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f2796-674b-4512-9258-e13745889543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:37.176011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-716f2796-674b-4512-9258-e13745889543 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:37.320480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f2796-674b-4512-9258-e13745889543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:37.323037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f2796-674b-4512-9258-e13745889543 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:37.341317 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 980/1965] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:37.405233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7cfb9-909c-424f-920e-d8bbde9ad308 service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:37.423937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:37.425033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:37.425589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:37.426073 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38 admin admin] PATH_INFO: `/v3/projects/75d36fe7dc5b41b5a281cbcab40743a9` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:37.427633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75d36fe7dc5b41b5a281cbcab40743a9)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:37.431726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:37.614062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75d36fe7dc5b41b5a281cbcab40743a9', 'request_id': 'req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:37.624033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75d36fe7dc5b41b5a281cbcab40743a9', 'request_id': 'req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:37.639939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75d36fe7dc5b41b5a281cbcab40743a9', 'request_id': 'req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:37.670692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38 admin admin] The token cache is being invalidate because project 75d36fe7dc5b41b5a281cbcab40743a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:37.671434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:37.673056 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e843b97d-77f1-4cd9-828e-9cf76e5b5c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:37.673903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 986/1966] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:37 2022] DELETE /identity/v3/projects/75d36fe7dc5b41b5a281cbcab40743a9 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:37.678659 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5c0bb-2a03-4ea8-9ee6-ca82a5ed00cc None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:37.679252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5c0bb-2a03-4ea8-9ee6-ca82a5ed00cc None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:37.679252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5c0bb-2a03-4ea8-9ee6-ca82a5ed00cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:37.706100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-9ee5c0bb-2a03-4ea8-9ee6-ca82a5ed00cc None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:37.707439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-9ee5c0bb-2a03-4ea8-9ee6-ca82a5ed00cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:37.729906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:37.795059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 981/1967] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:37.802353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d069e51-364f-4dc8-9de3-ff4ff7ef9d77 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:37.802633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d069e51-364f-4dc8-9de3-ff4ff7ef9d77 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:37.802909 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d069e51-364f-4dc8-9de3-ff4ff7ef9d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:37.827036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-7d069e51-364f-4dc8-9de3-ff4ff7ef9d77 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:37.828381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7d069e51-364f-4dc8-9de3-ff4ff7ef9d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:37.842730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 987/1968] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:37.847584 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5c0bb-2a03-4ea8-9ee6-ca82a5ed00cc None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:37.859460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5c0bb-2a03-4ea8-9ee6-ca82a5ed00cc None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:37.864853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9979d-14a2-4af9-b57a-7821bfbd02ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:37.865493 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9979d-14a2-4af9-b57a-7821bfbd02ac admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:37.865786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9979d-14a2-4af9-b57a-7821bfbd02ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:37.866040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9979d-14a2-4af9-b57a-7821bfbd02ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:37.866694 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9979d-14a2-4af9-b57a-7821bfbd02ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:17:37.866994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9979d-14a2-4af9-b57a-7821bfbd02ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:37.868764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9979d-14a2-4af9-b57a-7821bfbd02ac admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:37.883163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 982/1969] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:17:37.891331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d069e51-364f-4dc8-9de3-ff4ff7ef9d77 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:37.901444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d069e51-364f-4dc8-9de3-ff4ff7ef9d77 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:37.911412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceafc6a-7eeb-4d39-9731-2e12b05e1f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:37.913389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceafc6a-7eeb-4d39-9731-2e12b05e1f0a admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:37.913773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceafc6a-7eeb-4d39-9731-2e12b05e1f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:37.914068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceafc6a-7eeb-4d39-9731-2e12b05e1f0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:37.914736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceafc6a-7eeb-4d39-9731-2e12b05e1f0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:37.917047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceafc6a-7eeb-4d39-9731-2e12b05e1f0a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:37.961113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 988/1970] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:37 2022] POST /identity/v3/projects => generated 393 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:37.961698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9979d-14a2-4af9-b57a-7821bfbd02ac admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:37.975140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9979d-14a2-4af9-b57a-7821bfbd02ac admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:38.053213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b033bb-f0bb-42c0-8ca0-6d92f05f4f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:38.053952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b033bb-f0bb-42c0-8ca0-6d92f05f4f3b admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:38.054304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b033bb-f0bb-42c0-8ca0-6d92f05f4f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:38.054844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b033bb-f0bb-42c0-8ca0-6d92f05f4f3b admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:38.056179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b033bb-f0bb-42c0-8ca0-6d92f05f4f3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:38.058485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b033bb-f0bb-42c0-8ca0-6d92f05f4f3b admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:38.104386 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 983/1971] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:37 2022] POST /identity/v3/users => generated 297 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:17:38.108926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceafc6a-7eeb-4d39-9731-2e12b05e1f0a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:38.120740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceafc6a-7eeb-4d39-9731-2e12b05e1f0a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:38.125978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522a2e4-8413-4204-a6ee-07242c884785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:38.126854 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522a2e4-8413-4204-a6ee-07242c884785 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:38.127172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522a2e4-8413-4204-a6ee-07242c884785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:38.127456 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522a2e4-8413-4204-a6ee-07242c884785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:38.128195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522a2e4-8413-4204-a6ee-07242c884785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:38.129921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522a2e4-8413-4204-a6ee-07242c884785 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:38.142349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 989/1972] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:38.146789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b033bb-f0bb-42c0-8ca0-6d92f05f4f3b admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:38.158865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b033bb-f0bb-42c0-8ca0-6d92f05f4f3b admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:38.168551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c9b31-8484-4604-a35a-ddce6c5261d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:38.168551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c9b31-8484-4604-a35a-ddce6c5261d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:38.168551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c9b31-8484-4604-a35a-ddce6c5261d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:38.168551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c9b31-8484-4604-a35a-ddce6c5261d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:38.169171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c9b31-8484-4604-a35a-ddce6c5261d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:38.170712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c9b31-8484-4604-a35a-ddce6c5261d2 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:38.185171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 984/1973] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:38.189744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522a2e4-8413-4204-a6ee-07242c884785 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:38.202338 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522a2e4-8413-4204-a6ee-07242c884785 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:38.207684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3ef92-3277-48d2-a50c-5bb0904bc3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:38.208382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3ef92-3277-48d2-a50c-5bb0904bc3fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:38.208662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3ef92-3277-48d2-a50c-5bb0904bc3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:38.208937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3ef92-3277-48d2-a50c-5bb0904bc3fb admin admin] PATH_INFO: `/v3/projects/0d19adb2d998448aad6acdc3c854f9ed/users/1680a9cf7af94e1987501632a63b9246/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:38.227974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3ef92-3277-48d2-a50c-5bb0904bc3fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d19adb2d998448aad6acdc3c854f9ed, user_id=1680a9cf7af94e1987501632a63b9246, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:38.237547 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3ef92-3277-48d2-a50c-5bb0904bc3fb admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:38.247685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:38.257213 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 990/1974] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:38 2022] PUT /identity/v3/projects/0d19adb2d998448aad6acdc3c854f9ed/users/1680a9cf7af94e1987501632a63b9246/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:38.276924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e818ac-58d4-45d7-9f38-ac6e008adb47 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:38.277601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e818ac-58d4-45d7-9f38-ac6e008adb47 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:38.278003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e818ac-58d4-45d7-9f38-ac6e008adb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:38.315542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-28e818ac-58d4-45d7-9f38-ac6e008adb47 None None] MFA Rules not processed for user `1680a9cf7af94e1987501632a63b9246`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:38.317284 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-28e818ac-58d4-45d7-9f38-ac6e008adb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:38.391841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 985/1975] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:38.396508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3ef92-3277-48d2-a50c-5bb0904bc3fb admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:38.471036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3ef92-3277-48d2-a50c-5bb0904bc3fb admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:38.476636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe2d51-cb50-4eb6-9b2b-f2850660a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:38.477267 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe2d51-cb50-4eb6-9b2b-f2850660a6de admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:38.477590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe2d51-cb50-4eb6-9b2b-f2850660a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:38.477861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe2d51-cb50-4eb6-9b2b-f2850660a6de admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:38.478520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe2d51-cb50-4eb6-9b2b-f2850660a6de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:38.480914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe2d51-cb50-4eb6-9b2b-f2850660a6de admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:38.513513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 991/1976] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:38 2022] POST /identity/v3/projects => generated 393 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:38.517682 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e818ac-58d4-45d7-9f38-ac6e008adb47 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:38.530757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e818ac-58d4-45d7-9f38-ac6e008adb47 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:38.548575 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:38.600209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e0909-517d-4c0b-9d22-e60a80fba324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:38.605020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e0909-517d-4c0b-9d22-e60a80fba324 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:38.605020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e0909-517d-4c0b-9d22-e60a80fba324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:38.605020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e0909-517d-4c0b-9d22-e60a80fba324 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:38.605020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e0909-517d-4c0b-9d22-e60a80fba324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:38.605020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e0909-517d-4c0b-9d22-e60a80fba324 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:38.635911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 986/1977] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:38 2022] POST /identity/v3/users => generated 303 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:38.637168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe2d51-cb50-4eb6-9b2b-f2850660a6de admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:38.647742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe2d51-cb50-4eb6-9b2b-f2850660a6de admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:38.653350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a8cf7-9c33-45ba-99ba-03b01e5a3316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:38.653974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a8cf7-9c33-45ba-99ba-03b01e5a3316 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:38.654255 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a8cf7-9c33-45ba-99ba-03b01e5a3316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:38.654537 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a8cf7-9c33-45ba-99ba-03b01e5a3316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:38.655323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a8cf7-9c33-45ba-99ba-03b01e5a3316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:38.656935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a8cf7-9c33-45ba-99ba-03b01e5a3316 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:38.675154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 992/1978] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:38.678526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e0909-517d-4c0b-9d22-e60a80fba324 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:38.690732 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e0909-517d-4c0b-9d22-e60a80fba324 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:38.696027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f46f90-91c1-48df-a8cf-d2192745d920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:38.697325 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f46f90-91c1-48df-a8cf-d2192745d920 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:38.697601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f46f90-91c1-48df-a8cf-d2192745d920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:38.697880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f46f90-91c1-48df-a8cf-d2192745d920 admin admin] PATH_INFO: `/v3/projects/0270cf9b4a694623a77dfd1d284cd626/users/280ebdad74404e929d4c82307d661552/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:38.719365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f46f90-91c1-48df-a8cf-d2192745d920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0270cf9b4a694623a77dfd1d284cd626, user_id=280ebdad74404e929d4c82307d661552, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:38.720042 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f46f90-91c1-48df-a8cf-d2192745d920 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:38.734638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 987/1979] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:38 2022] PUT /identity/v3/projects/0270cf9b4a694623a77dfd1d284cd626/users/280ebdad74404e929d4c82307d661552/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:38.738208 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a8cf7-9c33-45ba-99ba-03b01e5a3316 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:38.749284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a8cf7-9c33-45ba-99ba-03b01e5a3316 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:38.823887 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94977927-858d-4f47-becb-11eada42b258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:38.824668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94977927-858d-4f47-becb-11eada42b258 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:38.824980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94977927-858d-4f47-becb-11eada42b258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:38.825251 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94977927-858d-4f47-becb-11eada42b258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:38.825906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94977927-858d-4f47-becb-11eada42b258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:38.827536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94977927-858d-4f47-becb-11eada42b258 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:38.837997 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 993/1980] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:38.842275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f46f90-91c1-48df-a8cf-d2192745d920 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:38.853059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f46f90-91c1-48df-a8cf-d2192745d920 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:38.858018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d3b4b-8281-43b1-9218-1be62aff6ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:38.858939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3b4b-8281-43b1-9218-1be62aff6ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:38.859272 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3b4b-8281-43b1-9218-1be62aff6ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:38.859547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3b4b-8281-43b1-9218-1be62aff6ef8 admin admin] PATH_INFO: `/v3/projects/0270cf9b4a694623a77dfd1d284cd626/users/280ebdad74404e929d4c82307d661552/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:38.861502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d3b4b-8281-43b1-9218-1be62aff6ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0270cf9b4a694623a77dfd1d284cd626, user_id=280ebdad74404e929d4c82307d661552, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:38.864341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d3b4b-8281-43b1-9218-1be62aff6ef8 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:38.876833 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 988/1981] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:38 2022] PUT /identity/v3/projects/0270cf9b4a694623a77dfd1d284cd626/users/280ebdad74404e929d4c82307d661552/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:38.888153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72262b-093a-43e4-8494-015718ad9cbd None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:38.888443 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72262b-093a-43e4-8494-015718ad9cbd None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:38.888727 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72262b-093a-43e4-8494-015718ad9cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:38.919336 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-cf72262b-093a-43e4-8494-015718ad9cbd None None] MFA Rules not processed for user `280ebdad74404e929d4c82307d661552`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:38.920558 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-cf72262b-093a-43e4-8494-015718ad9cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:38.997294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:39.010328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 994/1982] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:38 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:39.023059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d3b4b-8281-43b1-9218-1be62aff6ef8 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:39.024835 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-bc1d3b4b-8281-43b1-9218-1be62aff6ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:39.090430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d3b4b-8281-43b1-9218-1be62aff6ef8 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:39.095635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457fc79-ab92-4c64-b54f-5e4577dee00a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:39.096266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457fc79-ab92-4c64-b54f-5e4577dee00a service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:39.096547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457fc79-ab92-4c64-b54f-5e4577dee00a service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:39.096822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457fc79-ab92-4c64-b54f-5e4577dee00a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:39.106656 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457fc79-ab92-4c64-b54f-5e4577dee00a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:39.108331 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457fc79-ab92-4c64-b54f-5e4577dee00a service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:39.120286 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 989/1983] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:39 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:39.173448 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72262b-093a-43e4-8494-015718ad9cbd None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:39.174805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-cf72262b-093a-43e4-8494-015718ad9cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:39.188895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72262b-093a-43e4-8494-015718ad9cbd None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:39.192742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678f15b-48c7-4813-9d78-2131be9cf0bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:39.193360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678f15b-48c7-4813-9d78-2131be9cf0bc service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:39.193643 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678f15b-48c7-4813-9d78-2131be9cf0bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:39.193917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678f15b-48c7-4813-9d78-2131be9cf0bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:39.271352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678f15b-48c7-4813-9d78-2131be9cf0bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:39.273301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678f15b-48c7-4813-9d78-2131be9cf0bc service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:39.284149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 995/1984] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:39 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:49.114695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=61,keystone:INSERT=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:49.278920 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=31 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:49.721170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457fc79-ab92-4c64-b54f-5e4577dee00a service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:49.724207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0457fc79-ab92-4c64-b54f-5e4577dee00a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:49.815697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457fc79-ab92-4c64-b54f-5e4577dee00a service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:49.822123 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:49.823252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:49.823400 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:49.823643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] PATH_INFO: `/v3/users/3835a910d73146d5933b2b72e6d02ba7` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:49.824878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3835a910d73146d5933b2b72e6d02ba7)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:49.826832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:49.911321 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3835a910d73146d5933b2b72e6d02ba7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:49.923278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3835a910d73146d5933b2b72e6d02ba7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:49.923902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3835a910d73146d5933b2b72e6d02ba7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:49.924539 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3835a910d73146d5933b2b72e6d02ba7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:49.941713 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3835a910d73146d5933b2b72e6d02ba7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:49.958518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 990/1985] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:49 2022] DELETE /identity/v3/users/3835a910d73146d5933b2b72e6d02ba7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:49.963137 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25d5e4-030c-4b33-bb63-11145cb8a525 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:49.963426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25d5e4-030c-4b33-bb63-11145cb8a525 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:49.963701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25d5e4-030c-4b33-bb63-11145cb8a525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:49.987090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-9c25d5e4-030c-4b33-bb63-11145cb8a525 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:49.988722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9c25d5e4-030c-4b33-bb63-11145cb8a525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:50.073657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:50.073657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:50.087257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 996/1986] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:50.088096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f9222-67d9-4e3d-9a8d-68192a802ed0 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:50.097134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:50.097991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:50.098275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:50.098551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] PATH_INFO: `/v3/users/1680a9cf7af94e1987501632a63b9246` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:50.098961 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25d5e4-030c-4b33-bb63-11145cb8a525 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:50.099963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1680a9cf7af94e1987501632a63b9246)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:50.102523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:50.104759 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9c25d5e4-030c-4b33-bb63-11145cb8a525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:50.184788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1680a9cf7af94e1987501632a63b9246'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:50.192440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25d5e4-030c-4b33-bb63-11145cb8a525 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:50.194006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1680a9cf7af94e1987501632a63b9246'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:50.194754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1680a9cf7af94e1987501632a63b9246'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:50.195591 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1680a9cf7af94e1987501632a63b9246'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:50.197225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be21c6a-b873-486c-9b96-2018a853290a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:50.197853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be21c6a-b873-486c-9b96-2018a853290a service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:50.198134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be21c6a-b873-486c-9b96-2018a853290a service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:50.198408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be21c6a-b873-486c-9b96-2018a853290a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:50.200786 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5be21c6a-b873-486c-9b96-2018a853290a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:50.214923 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1680a9cf7af94e1987501632a63b9246'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:50.220308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:50.232113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be21c6a-b873-486c-9b96-2018a853290a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:50.234756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be21c6a-b873-486c-9b96-2018a853290a service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:50.247474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 997/1987] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:50.251731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 991/1988] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:50 2022] DELETE /identity/v3/users/1680a9cf7af94e1987501632a63b9246 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:50.256722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c685e-c98e-4399-bdfb-eceb56a16931 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:50.257490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c685e-c98e-4399-bdfb-eceb56a16931 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:50.257490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c685e-c98e-4399-bdfb-eceb56a16931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:50.285209 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-3a7c685e-c98e-4399-bdfb-eceb56a16931 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:50.286979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3a7c685e-c98e-4399-bdfb-eceb56a16931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:50.347158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:50.348938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:50.381548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 998/1989] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:50.394648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c685e-c98e-4399-bdfb-eceb56a16931 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:50.397367 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3a7c685e-c98e-4399-bdfb-eceb56a16931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:50.436735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51920c-1d7d-494c-b92d-1ec252e264bd admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:50.443057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:50.443962 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:50.444166 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:50.444563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] PATH_INFO: `/v3/users/19fcbd1f8f3a4dc98f5ded776563b5f3` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:50.446539 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] RBAC: Authorizing `identity:delete_user(user_id=19fcbd1f8f3a4dc98f5ded776563b5f3)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:50.448758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:50.485710 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:50.499879 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c685e-c98e-4399-bdfb-eceb56a16931 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:50.507501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef042c-7535-4ec1-8559-629cb0707774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:50.508395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef042c-7535-4ec1-8559-629cb0707774 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:50.509510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef042c-7535-4ec1-8559-629cb0707774 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:50.509948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef042c-7535-4ec1-8559-629cb0707774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:50.521188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef042c-7535-4ec1-8559-629cb0707774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:50.521441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19fcbd1f8f3a4dc98f5ded776563b5f3'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:50.523582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef042c-7535-4ec1-8559-629cb0707774 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:50.540333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 999/1990] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:50.542057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19fcbd1f8f3a4dc98f5ded776563b5f3'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:50.543005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19fcbd1f8f3a4dc98f5ded776563b5f3'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:50.544467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19fcbd1f8f3a4dc98f5ded776563b5f3'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:50.583311 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19fcbd1f8f3a4dc98f5ded776563b5f3'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:50.610908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 992/1991] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:50 2022] DELETE /identity/v3/users/19fcbd1f8f3a4dc98f5ded776563b5f3 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:50.615935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab8d89-b525-4124-8db4-33332f19ed9c None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:50.616476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab8d89-b525-4124-8db4-33332f19ed9c None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:50.616982 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab8d89-b525-4124-8db4-33332f19ed9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:50.644948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-84ab8d89-b525-4124-8db4-33332f19ed9c None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:50.647325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-84ab8d89-b525-4124-8db4-33332f19ed9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:50.746252 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1000/1992] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:50.754074 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:50.755783 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:50.767414 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab8d89-b525-4124-8db4-33332f19ed9c None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:50.771700 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-84ab8d89-b525-4124-8db4-33332f19ed9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:50.814499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9835bd-4848-4d71-9615-b9bf5fff212e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:50.825368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65ad46-62ce-467b-8fea-cc748387773e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:50.829807 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65ad46-62ce-467b-8fea-cc748387773e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:50.830172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65ad46-62ce-467b-8fea-cc748387773e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:50.830450 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65ad46-62ce-467b-8fea-cc748387773e admin admin] PATH_INFO: `/v3/projects/92296969740f43bf97f12b77fa4ebde1` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:50.832804 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65ad46-62ce-467b-8fea-cc748387773e admin admin] RBAC: Authorizing `identity:delete_project(project_id=92296969740f43bf97f12b77fa4ebde1)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:50.834763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65ad46-62ce-467b-8fea-cc748387773e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:50.935165 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-0d65ad46-62ce-467b-8fea-cc748387773e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92296969740f43bf97f12b77fa4ebde1', 'request_id': 'req-0d65ad46-62ce-467b-8fea-cc748387773e'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:50.982136 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab8d89-b525-4124-8db4-33332f19ed9c None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:50.983595 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-0d65ad46-62ce-467b-8fea-cc748387773e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92296969740f43bf97f12b77fa4ebde1', 'request_id': 'req-0d65ad46-62ce-467b-8fea-cc748387773e'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:50.994686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-0d65ad46-62ce-467b-8fea-cc748387773e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92296969740f43bf97f12b77fa4ebde1', 'request_id': 'req-0d65ad46-62ce-467b-8fea-cc748387773e'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:50.996786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:51.023939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-0d65ad46-62ce-467b-8fea-cc748387773e admin admin] The token cache is being invalidate because project 92296969740f43bf97f12b77fa4ebde1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:51.025567 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-0d65ad46-62ce-467b-8fea-cc748387773e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.026499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-0d65ad46-62ce-467b-8fea-cc748387773e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.035209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 993/1993] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:50 2022] DELETE /identity/v3/projects/92296969740f43bf97f12b77fa4ebde1 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:51.037063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfab05-6f9f-4e8a-82ad-75ee21d2a375 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:51.037356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfab05-6f9f-4e8a-82ad-75ee21d2a375 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:51.037640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfab05-6f9f-4e8a-82ad-75ee21d2a375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:51.066402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-a1dfab05-6f9f-4e8a-82ad-75ee21d2a375 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:51.071251 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278c555-51dd-4bd7-929c-8e5fbdcc4147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:51.072149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278c555-51dd-4bd7-929c-8e5fbdcc4147 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:51.072702 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278c555-51dd-4bd7-929c-8e5fbdcc4147 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:51.073018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278c555-51dd-4bd7-929c-8e5fbdcc4147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:51.073911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a1dfab05-6f9f-4e8a-82ad-75ee21d2a375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:51.083370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e278c555-51dd-4bd7-929c-8e5fbdcc4147 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:51.193234 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 994/1994] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:51.193997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfab05-6f9f-4e8a-82ad-75ee21d2a375 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:51.196516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a1dfab05-6f9f-4e8a-82ad-75ee21d2a375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:51.212275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfab05-6f9f-4e8a-82ad-75ee21d2a375 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:51.217924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:51.218674 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:51.219021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:51.219572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] PATH_INFO: `/v3/projects/0d19adb2d998448aad6acdc3c854f9ed` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:51.220690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d19adb2d998448aad6acdc3c854f9ed)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:51.222994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:51.224370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:51.230397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278c555-51dd-4bd7-929c-8e5fbdcc4147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:51.231302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278c555-51dd-4bd7-929c-8e5fbdcc4147 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:51.246022 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1001/1995] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:51.248702 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf84e23-4ee0-496c-96dd-3c4ad9667d0a None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:51.249204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf84e23-4ee0-496c-96dd-3c4ad9667d0a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:51.251391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf84e23-4ee0-496c-96dd-3c4ad9667d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:51.285101 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-faf84e23-4ee0-496c-96dd-3c4ad9667d0a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:51.287435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-faf84e23-4ee0-496c-96dd-3c4ad9667d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:51.292561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d19adb2d998448aad6acdc3c854f9ed', 'request_id': 'req-1ee309fc-d627-426d-9258-b73712080769'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.302934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d19adb2d998448aad6acdc3c854f9ed', 'request_id': 'req-1ee309fc-d627-426d-9258-b73712080769'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.312328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d19adb2d998448aad6acdc3c854f9ed', 'request_id': 'req-1ee309fc-d627-426d-9258-b73712080769'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.326118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] The token cache is being invalidate because project 0d19adb2d998448aad6acdc3c854f9ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:51.326118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.326925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.328824 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 995/1996] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:51 2022] DELETE /identity/v3/projects/0d19adb2d998448aad6acdc3c854f9ed => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:51.333080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:51.335371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:51.407701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1002/1997] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:51.413054 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf84e23-4ee0-496c-96dd-3c4ad9667d0a None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:51.415702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee309fc-d627-426d-9258-b73712080769 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:51.422769 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:51.423556 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:51.423836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:51.424107 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] PATH_INFO: `/v3/users/280ebdad74404e929d4c82307d661552` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:51.425468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=280ebdad74404e929d4c82307d661552)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:51.425741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf84e23-4ee0-496c-96dd-3c4ad9667d0a None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:51.427456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:51.431880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f899f8-700e-42ea-a80f-9c344818c3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:51.433319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f899f8-700e-42ea-a80f-9c344818c3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:51.433752 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f899f8-700e-42ea-a80f-9c344818c3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:51.435013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f899f8-700e-42ea-a80f-9c344818c3c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:51.435609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f899f8-700e-42ea-a80f-9c344818c3c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:17:51.435979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f899f8-700e-42ea-a80f-9c344818c3c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:51.438328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f899f8-700e-42ea-a80f-9c344818c3c4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:51.462563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:51.467309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1003/1998] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:17:51.468382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f899f8-700e-42ea-a80f-9c344818c3c4 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:51.482743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f899f8-700e-42ea-a80f-9c344818c3c4 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:51.489017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a9c17-caeb-49c8-ac30-aaf41db87bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:51.489993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a9c17-caeb-49c8-ac30-aaf41db87bef admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:51.490503 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a9c17-caeb-49c8-ac30-aaf41db87bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:51.491014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a9c17-caeb-49c8-ac30-aaf41db87bef admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:51.493278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a9c17-caeb-49c8-ac30-aaf41db87bef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:51.495875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a9c17-caeb-49c8-ac30-aaf41db87bef admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:51.507242 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '280ebdad74404e929d4c82307d661552'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.518951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '280ebdad74404e929d4c82307d661552'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.520417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '280ebdad74404e929d4c82307d661552'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.521758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '280ebdad74404e929d4c82307d661552'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.540561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1004/1999] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:51 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:51.546285 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a9c17-caeb-49c8-ac30-aaf41db87bef admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:51.547027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '280ebdad74404e929d4c82307d661552'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.551610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-490a9c17-caeb-49c8-ac30-aaf41db87bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:51.572491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 996/2000] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:51 2022] DELETE /identity/v3/users/280ebdad74404e929d4c82307d661552 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:51.646906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:51.649364 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:51.649750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a9c17-caeb-49c8-ac30-aaf41db87bef admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:51.760081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1514e-8ba2-48c2-947d-102e0acdf0c4 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:51.767134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:51.768203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:51.768891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:51.769642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] PATH_INFO: `/v3/projects/f4754913b2af4b04a2b740a251c33092` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:51.771498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4754913b2af4b04a2b740a251c33092)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:51.774730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:51.785235 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e42aed-4da2-41ce-a347-f0238679e451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:51.785941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e42aed-4da2-41ce-a347-f0238679e451 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:51.786286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e42aed-4da2-41ce-a347-f0238679e451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:51.786625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e42aed-4da2-41ce-a347-f0238679e451 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:51.787415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e42aed-4da2-41ce-a347-f0238679e451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:51.791158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e42aed-4da2-41ce-a347-f0238679e451 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:51.803615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:51.822508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1005/2001] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:51 2022] POST /identity/v3/users => generated 287 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:51.828342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e42aed-4da2-41ce-a347-f0238679e451 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:51.839878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4754913b2af4b04a2b740a251c33092', 'request_id': 'req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.844866 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e42aed-4da2-41ce-a347-f0238679e451 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:51.850742 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4754913b2af4b04a2b740a251c33092', 'request_id': 'req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.860577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4754913b2af4b04a2b740a251c33092', 'request_id': 'req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.875054 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] The token cache is being invalidate because project f4754913b2af4b04a2b740a251c33092 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:51.875920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.876711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:51.878763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 997/2002] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:51 2022] DELETE /identity/v3/projects/f4754913b2af4b04a2b740a251c33092 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:51.884967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:51.887075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:51.926261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:51.934209 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b97309-d982-4b44-90d6-a647073ff360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:51.934954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b97309-d982-4b44-90d6-a647073ff360 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:51.935550 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b97309-d982-4b44-90d6-a647073ff360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:51.935835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b97309-d982-4b44-90d6-a647073ff360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:51.936502 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b97309-d982-4b44-90d6-a647073ff360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:51.941373 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b97309-d982-4b44-90d6-a647073ff360 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:51.956975 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1006/2003] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:51.962775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b97309-d982-4b44-90d6-a647073ff360 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:51.965936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-03b97309-d982-4b44-90d6-a647073ff360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:51.990068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f5305-ccad-4fc8-9f8f-b403d1e1a149 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:51.997227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:51.997429 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:51.997542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:51.997794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b97309-d982-4b44-90d6-a647073ff360 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:51.997917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] PATH_INFO: `/v3/users/d1c4cb37918e4945a0b2812b381993c8` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:51.999139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c4cb37918e4945a0b2812b381993c8)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:52.001081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:52.020688 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b048f-4893-4e5a-8774-7436cd96c9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:52.021520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b048f-4893-4e5a-8774-7436cd96c9bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:52.021826 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b048f-4893-4e5a-8774-7436cd96c9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:52.022084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b048f-4893-4e5a-8774-7436cd96c9bf admin admin] PATH_INFO: `/v3/projects/0270cf9b4a694623a77dfd1d284cd626` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:52.024749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b048f-4893-4e5a-8774-7436cd96c9bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0270cf9b4a694623a77dfd1d284cd626)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:52.026659 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b048f-4893-4e5a-8774-7436cd96c9bf admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:52.095819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c4cb37918e4945a0b2812b381993c8'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:52.099182 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-040b048f-4893-4e5a-8774-7436cd96c9bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0270cf9b4a694623a77dfd1d284cd626', 'request_id': 'req-040b048f-4893-4e5a-8774-7436cd96c9bf'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:52.104321 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c4cb37918e4945a0b2812b381993c8'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:52.105213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c4cb37918e4945a0b2812b381993c8'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:52.105911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c4cb37918e4945a0b2812b381993c8'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:52.106978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-040b048f-4893-4e5a-8774-7436cd96c9bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0270cf9b4a694623a77dfd1d284cd626', 'request_id': 'req-040b048f-4893-4e5a-8774-7436cd96c9bf'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:52.117460 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-040b048f-4893-4e5a-8774-7436cd96c9bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0270cf9b4a694623a77dfd1d284cd626', 'request_id': 'req-040b048f-4893-4e5a-8774-7436cd96c9bf'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:52.124999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c4cb37918e4945a0b2812b381993c8'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:52.146122 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-040b048f-4893-4e5a-8774-7436cd96c9bf admin admin] The token cache is being invalidate because project 0270cf9b4a694623a77dfd1d284cd626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:52.147375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-040b048f-4893-4e5a-8774-7436cd96c9bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:52.148242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-040b048f-4893-4e5a-8774-7436cd96c9bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:52.150599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 998/2004] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:51 2022] DELETE /identity/v3/users/d1c4cb37918e4945a0b2812b381993c8 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:52.151673 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:52.154794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:52.155878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1007/2005] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:51 2022] DELETE /identity/v3/projects/0270cf9b4a694623a77dfd1d284cd626 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:52.157532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e854f-c689-4fa0-8ff5-61991fb2fd1d None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:52.157868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e854f-c689-4fa0-8ff5-61991fb2fd1d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:52.158147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e854f-c689-4fa0-8ff5-61991fb2fd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:52.187033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b39e854f-c689-4fa0-8ff5-61991fb2fd1d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:52.188556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b39e854f-c689-4fa0-8ff5-61991fb2fd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:52.260395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf92944-607f-4dac-b6e6-6aa31985ca78 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:52.268966 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5c58e-4e6d-425c-828f-7524c27bf33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:52.296604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5c58e-4e6d-425c-828f-7524c27bf33a admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:52.297296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5c58e-4e6d-425c-828f-7524c27bf33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:52.297610 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5c58e-4e6d-425c-828f-7524c27bf33a admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:52.298852 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5c58e-4e6d-425c-828f-7524c27bf33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:52.307076 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5c58e-4e6d-425c-828f-7524c27bf33a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:52.308598 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1008/2006] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:52.322195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66e8bd-42c3-41ba-b102-e4bac18f1299 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:52.323845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66e8bd-42c3-41ba-b102-e4bac18f1299 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:52.323845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66e8bd-42c3-41ba-b102-e4bac18f1299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:52.341678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 999/2007] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:52.346472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5c58e-4e6d-425c-828f-7524c27bf33a admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:52.350138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a1a5c58e-4e6d-425c-828f-7524c27bf33a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:52.368355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-bb66e8bd-42c3-41ba-b102-e4bac18f1299 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:52.369803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bb66e8bd-42c3-41ba-b102-e4bac18f1299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:52.380599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5c58e-4e6d-425c-828f-7524c27bf33a admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:52.386070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1009/2008] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:52.388251 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66e8bd-42c3-41ba-b102-e4bac18f1299 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:52.390789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8ee5d-4871-48de-a0fe-10b150baf79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:52.391718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8ee5d-4871-48de-a0fe-10b150baf79c admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:52.392146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8ee5d-4871-48de-a0fe-10b150baf79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:52.393128 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8ee5d-4871-48de-a0fe-10b150baf79c admin admin] PATH_INFO: `/v3/projects/6d3b344911654d3a83328d54b8feca0e/users/3f93418e3e264a1f8597f789ee888b4e/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:52.419801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8ee5d-4871-48de-a0fe-10b150baf79c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3b344911654d3a83328d54b8feca0e, user_id=3f93418e3e264a1f8597f789ee888b4e, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:52.421834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66e8bd-42c3-41ba-b102-e4bac18f1299 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:52.423020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8ee5d-4871-48de-a0fe-10b150baf79c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:52.437697 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cba344-c6e5-4a36-b8f0-7fbaf9c872eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:52.438328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cba344-c6e5-4a36-b8f0-7fbaf9c872eb admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:52.440141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cba344-c6e5-4a36-b8f0-7fbaf9c872eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:52.440429 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cba344-c6e5-4a36-b8f0-7fbaf9c872eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:52.442062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cba344-c6e5-4a36-b8f0-7fbaf9c872eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:17:52.444235 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cba344-c6e5-4a36-b8f0-7fbaf9c872eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:52.447832 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cba344-c6e5-4a36-b8f0-7fbaf9c872eb admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:52.453460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1000/2009] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:52 2022] PUT /identity/v3/projects/6d3b344911654d3a83328d54b8feca0e/users/3f93418e3e264a1f8597f789ee888b4e/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:52.463286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:52.463919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6f88f-b995-4233-97fc-38160a09b48f None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:52.463919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6f88f-b995-4233-97fc-38160a09b48f None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:52.463919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6f88f-b995-4233-97fc-38160a09b48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:52.480941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1010/2010] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:17:52.485784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cba344-c6e5-4a36-b8f0-7fbaf9c872eb admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:52.508016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cba344-c6e5-4a36-b8f0-7fbaf9c872eb admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:52.513632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-0ab6f88f-b995-4233-97fc-38160a09b48f None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:52.515197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0ab6f88f-b995-4233-97fc-38160a09b48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:52.548048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:52.624113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1001/2011] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:52.647559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cdbcc-f5f5-44e8-ad28-ae3b5fd72ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:52.649237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cdbcc-f5f5-44e8-ad28-ae3b5fd72ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:52.650177 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cdbcc-f5f5-44e8-ad28-ae3b5fd72ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:52.650814 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cdbcc-f5f5-44e8-ad28-ae3b5fd72ed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:52.652635 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cdbcc-f5f5-44e8-ad28-ae3b5fd72ed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:52.658176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6f88f-b995-4233-97fc-38160a09b48f None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:52.660635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0ab6f88f-b995-4233-97fc-38160a09b48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:52.664892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cdbcc-f5f5-44e8-ad28-ae3b5fd72ed9 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:52.721167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1011/2012] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:52 2022] POST /identity/v3/projects => generated 383 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:52.724814 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cdbcc-f5f5-44e8-ad28-ae3b5fd72ed9 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:52.785381 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6f88f-b995-4233-97fc-38160a09b48f None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:52.830589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cdbcc-f5f5-44e8-ad28-ae3b5fd72ed9 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:52.840944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1968e5-ece7-45a9-acb6-56ce5730b0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:52.841546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1968e5-ece7-45a9-acb6-56ce5730b0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:52.842117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1968e5-ece7-45a9-acb6-56ce5730b0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:52.842687 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1968e5-ece7-45a9-acb6-56ce5730b0a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:52.844145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1968e5-ece7-45a9-acb6-56ce5730b0a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:52.847969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1968e5-ece7-45a9-acb6-56ce5730b0a4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:52.880283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f02a18-499e-4b41-a47a-f97c3b5cc624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:52.880912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f02a18-499e-4b41-a47a-f97c3b5cc624 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:52.881186 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f02a18-499e-4b41-a47a-f97c3b5cc624 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:52.881455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f02a18-499e-4b41-a47a-f97c3b5cc624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:52.886026 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1012/2013] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:52 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:17:52.887192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1968e5-ece7-45a9-acb6-56ce5730b0a4 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:52.889490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3a1968e5-ece7-45a9-acb6-56ce5730b0a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:52.892404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f02a18-499e-4b41-a47a-f97c3b5cc624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:52.893863 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f02a18-499e-4b41-a47a-f97c3b5cc624 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:52.905999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1002/2014] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:17:52.906272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1968e5-ece7-45a9-acb6-56ce5730b0a4 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:52.907874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f02a18-499e-4b41-a47a-f97c3b5cc624 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:52.914612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:52.915291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:52.915880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:52.916168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] PATH_INFO: `/v3/projects/11fb23877ddd45f9981a8cd01907323c` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:52.917396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11fb23877ddd45f9981a8cd01907323c)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:52.917990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f02a18-499e-4b41-a47a-f97c3b5cc624 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:52.919704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:52.923294 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565bcd2-e027-4b11-88fe-b5a19924aa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:52.923960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565bcd2-e027-4b11-88fe-b5a19924aa6d admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:52.924301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565bcd2-e027-4b11-88fe-b5a19924aa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:52.925376 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565bcd2-e027-4b11-88fe-b5a19924aa6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:52.926299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565bcd2-e027-4b11-88fe-b5a19924aa6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:52.928173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565bcd2-e027-4b11-88fe-b5a19924aa6d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:52.943654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1003/2015] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:52.947755 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565bcd2-e027-4b11-88fe-b5a19924aa6d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:52.963476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565bcd2-e027-4b11-88fe-b5a19924aa6d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:52.971174 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40521a3f-954f-4633-81ad-884a2a74aa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:52.972170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40521a3f-954f-4633-81ad-884a2a74aa42 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:52.972506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40521a3f-954f-4633-81ad-884a2a74aa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:52.972844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40521a3f-954f-4633-81ad-884a2a74aa42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:52.973568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40521a3f-954f-4633-81ad-884a2a74aa42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:52.976072 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40521a3f-954f-4633-81ad-884a2a74aa42 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:52.995152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1004/2016] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:53.012514 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40521a3f-954f-4633-81ad-884a2a74aa42 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:53.020153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11fb23877ddd45f9981a8cd01907323c', 'request_id': 'req-19ad2400-ca53-4e4a-b411-7bef41b32755'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:53.028627 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40521a3f-954f-4633-81ad-884a2a74aa42 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:53.034175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11fb23877ddd45f9981a8cd01907323c', 'request_id': 'req-19ad2400-ca53-4e4a-b411-7bef41b32755'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:53.045813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11fb23877ddd45f9981a8cd01907323c', 'request_id': 'req-19ad2400-ca53-4e4a-b411-7bef41b32755'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:53.051996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:53.096768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] The token cache is being invalidate because project 11fb23877ddd45f9981a8cd01907323c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:53.097680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:53.100716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:53.105590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1013/2017] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:52 2022] DELETE /identity/v3/projects/11fb23877ddd45f9981a8cd01907323c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:53.112863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:53.120710 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:53.127843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:53.139129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2a0f2-65cc-4d56-9db3-b679159ee568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:53.139871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2a0f2-65cc-4d56-9db3-b679159ee568 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:53.141315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2a0f2-65cc-4d56-9db3-b679159ee568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:53.142224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2a0f2-65cc-4d56-9db3-b679159ee568 admin admin] PATH_INFO: `/v3/projects/0d072d876a2549f8bc538c0e20d8a033/users/4e2fa621eaf3452db62d884e65da7005/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:53.165571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2a0f2-65cc-4d56-9db3-b679159ee568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d072d876a2549f8bc538c0e20d8a033, user_id=4e2fa621eaf3452db62d884e65da7005, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:53.169145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2a0f2-65cc-4d56-9db3-b679159ee568 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:53.204206 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1005/2018] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:53 2022] PUT /identity/v3/projects/0d072d876a2549f8bc538c0e20d8a033/users/4e2fa621eaf3452db62d884e65da7005/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:53.205860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24425fd-d92f-4ecc-a420-56f205f0844d None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:53.206146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24425fd-d92f-4ecc-a420-56f205f0844d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:53.206421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24425fd-d92f-4ecc-a420-56f205f0844d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:53.257018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f24425fd-d92f-4ecc-a420-56f205f0844d None None] MFA Rules not processed for user `4e2fa621eaf3452db62d884e65da7005`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:53.258350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f24425fd-d92f-4ecc-a420-56f205f0844d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:53.297495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad2400-ca53-4e4a-b411-7bef41b32755 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:53.395445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1006/2019] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:53.401197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24425fd-d92f-4ecc-a420-56f205f0844d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:53.404737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f24425fd-d92f-4ecc-a420-56f205f0844d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:53.415989 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:53.416827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:53.417002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:53.417543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] PATH_INFO: `/v3/users/5c496a451d7b4bb19fec448cb6d5ba59` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:53.419235 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c496a451d7b4bb19fec448cb6d5ba59)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:53.421929 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:53.508550 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c496a451d7b4bb19fec448cb6d5ba59'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:53.528347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24425fd-d92f-4ecc-a420-56f205f0844d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:53.530031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c496a451d7b4bb19fec448cb6d5ba59'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:53.541279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1906a7-1025-47dc-ba2b-e34a14d428c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:53.542306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c496a451d7b4bb19fec448cb6d5ba59'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:53.542754 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c496a451d7b4bb19fec448cb6d5ba59'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:53.542880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1906a7-1025-47dc-ba2b-e34a14d428c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:53.543350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1906a7-1025-47dc-ba2b-e34a14d428c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:53.544294 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1906a7-1025-47dc-ba2b-e34a14d428c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:53.544423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c496a451d7b4bb19fec448cb6d5ba59'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:53.544790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1906a7-1025-47dc-ba2b-e34a14d428c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:53.547026 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1906a7-1025-47dc-ba2b-e34a14d428c1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:53.578103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1014/2020] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:53 2022] DELETE /identity/v3/users/5c496a451d7b4bb19fec448cb6d5ba59 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:53.602664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1007/2021] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:53 2022] POST /identity/v3/projects => generated 383 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:53.604343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:53.606714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:53.637485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:53.752399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5d151-0377-4cce-8b07-fbf8787b6083 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:53.801018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1906a7-1025-47dc-ba2b-e34a14d428c1 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:53.803135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-fa1906a7-1025-47dc-ba2b-e34a14d428c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:53.881972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f9908-0c99-482c-9d64-8bfa2f47e368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:53.883077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f9908-0c99-482c-9d64-8bfa2f47e368 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:53.883600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f9908-0c99-482c-9d64-8bfa2f47e368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:53.884078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f9908-0c99-482c-9d64-8bfa2f47e368 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:53.885219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f9908-0c99-482c-9d64-8bfa2f47e368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:53.889314 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f9908-0c99-482c-9d64-8bfa2f47e368 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:53.896027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:53.935878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1906a7-1025-47dc-ba2b-e34a14d428c1 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:53.946361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cba62d-f1b3-4f0a-ad14-0c6304c9f6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:53.948301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cba62d-f1b3-4f0a-ad14-0c6304c9f6a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:53.948965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cba62d-f1b3-4f0a-ad14-0c6304c9f6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:53.950203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cba62d-f1b3-4f0a-ad14-0c6304c9f6a7 admin admin] PATH_INFO: `/v3/users/1a40559b69e141cb94d0420e4a8405be` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:53.957323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1015/2022] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:53 2022] POST /identity/v3/users => generated 298 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:53.958021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cba62d-f1b3-4f0a-ad14-0c6304c9f6a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a40559b69e141cb94d0420e4a8405be)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:53.962361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cba62d-f1b3-4f0a-ad14-0c6304c9f6a7 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:53.962728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f9908-0c99-482c-9d64-8bfa2f47e368 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:53.983563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f9908-0c99-482c-9d64-8bfa2f47e368 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:53.992953 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c18e77-986d-410c-b392-7e19dfeb4bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:53.994052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c18e77-986d-410c-b392-7e19dfeb4bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:53.994395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c18e77-986d-410c-b392-7e19dfeb4bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:53.994862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c18e77-986d-410c-b392-7e19dfeb4bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:53.996235 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c18e77-986d-410c-b392-7e19dfeb4bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:54.000070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c18e77-986d-410c-b392-7e19dfeb4bd0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:54.022146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1016/2023] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:54.030694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c18e77-986d-410c-b392-7e19dfeb4bd0 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:54.052945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c18e77-986d-410c-b392-7e19dfeb4bd0 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:54.061955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8fc78f-dd8f-436a-a0cb-e855ae4895b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:54.063543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fc78f-dd8f-436a-a0cb-e855ae4895b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:54.063811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fc78f-dd8f-436a-a0cb-e855ae4895b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:54.064285 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fc78f-dd8f-436a-a0cb-e855ae4895b9 admin admin] PATH_INFO: `/v3/projects/53e5b8ce762340618890a12d9697853f/users/1eb4a1ce172e41c6967b003598247be1/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:54.090318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-92cba62d-f1b3-4f0a-ad14-0c6304c9f6a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a40559b69e141cb94d0420e4a8405be'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:54.108863 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-92cba62d-f1b3-4f0a-ad14-0c6304c9f6a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a40559b69e141cb94d0420e4a8405be'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:54.110248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-92cba62d-f1b3-4f0a-ad14-0c6304c9f6a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a40559b69e141cb94d0420e4a8405be'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:54.111778 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-92cba62d-f1b3-4f0a-ad14-0c6304c9f6a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a40559b69e141cb94d0420e4a8405be'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:54.119962 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8fc78f-dd8f-436a-a0cb-e855ae4895b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e5b8ce762340618890a12d9697853f, user_id=1eb4a1ce172e41c6967b003598247be1, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:54.126889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8fc78f-dd8f-436a-a0cb-e855ae4895b9 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:54.141104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-92cba62d-f1b3-4f0a-ad14-0c6304c9f6a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a40559b69e141cb94d0420e4a8405be'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:54.153101 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1017/2024] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:54 2022] PUT /identity/v3/projects/53e5b8ce762340618890a12d9697853f/users/1eb4a1ce172e41c6967b003598247be1/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:54.157139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8fc78f-dd8f-436a-a0cb-e855ae4895b9 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:54.160142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ee8fc78f-dd8f-436a-a0cb-e855ae4895b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:54.171323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1008/2025] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:53 2022] DELETE /identity/v3/users/1a40559b69e141cb94d0420e4a8405be => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:54.176617 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af87309-7a0d-4da6-8fa2-86afdc19df35 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:54.177257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af87309-7a0d-4da6-8fa2-86afdc19df35 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:54.178217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af87309-7a0d-4da6-8fa2-86afdc19df35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:54.223592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-7af87309-7a0d-4da6-8fa2-86afdc19df35 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:54.244912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7af87309-7a0d-4da6-8fa2-86afdc19df35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:54.277034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8fc78f-dd8f-436a-a0cb-e855ae4895b9 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:54.309240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:54.347470 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4623f-c910-498c-8eb4-9ba0881e5af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:54.348264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4623f-c910-498c-8eb4-9ba0881e5af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:54.348599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4623f-c910-498c-8eb4-9ba0881e5af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:54.348877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4623f-c910-498c-8eb4-9ba0881e5af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:54.349728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4623f-c910-498c-8eb4-9ba0881e5af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:54.351617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4623f-c910-498c-8eb4-9ba0881e5af4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:54.368218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1018/2026] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:17:54.372126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4623f-c910-498c-8eb4-9ba0881e5af4 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:54.384860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4623f-c910-498c-8eb4-9ba0881e5af4 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:54.396942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479f6c0-106f-4fdd-a281-cb002acf8b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:54.397688 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479f6c0-106f-4fdd-a281-cb002acf8b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:54.398042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479f6c0-106f-4fdd-a281-cb002acf8b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:54.398390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479f6c0-106f-4fdd-a281-cb002acf8b46 admin admin] PATH_INFO: `/v3/projects/53e5b8ce762340618890a12d9697853f/users/1eb4a1ce172e41c6967b003598247be1/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:54.412170 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479f6c0-106f-4fdd-a281-cb002acf8b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e5b8ce762340618890a12d9697853f, user_id=1eb4a1ce172e41c6967b003598247be1, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:54.417903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479f6c0-106f-4fdd-a281-cb002acf8b46 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:54.419451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1009/2027] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:54.421107 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af87309-7a0d-4da6-8fa2-86afdc19df35 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:54.422907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7af87309-7a0d-4da6-8fa2-86afdc19df35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:54.437257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af87309-7a0d-4da6-8fa2-86afdc19df35 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:54.445057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e74855-d3f6-4303-a991-ca9a98215d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:54.446307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e74855-d3f6-4303-a991-ca9a98215d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:54.446307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e74855-d3f6-4303-a991-ca9a98215d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:54.446624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e74855-d3f6-4303-a991-ca9a98215d07 admin admin] PATH_INFO: `/v3/projects/9b6810fa0c024f4e9002a7f519d2c111` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:54.448415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e74855-d3f6-4303-a991-ca9a98215d07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6810fa0c024f4e9002a7f519d2c111)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:54.450766 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e74855-d3f6-4303-a991-ca9a98215d07 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:54.473063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1019/2028] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:54 2022] PUT /identity/v3/projects/53e5b8ce762340618890a12d9697853f/users/1eb4a1ce172e41c6967b003598247be1/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:54.476057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479f6c0-106f-4fdd-a281-cb002acf8b46 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:54.483152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7479f6c0-106f-4fdd-a281-cb002acf8b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:54.509932 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-46e74855-d3f6-4303-a991-ca9a98215d07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6810fa0c024f4e9002a7f519d2c111', 'request_id': 'req-46e74855-d3f6-4303-a991-ca9a98215d07'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:54.523854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-46e74855-d3f6-4303-a991-ca9a98215d07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6810fa0c024f4e9002a7f519d2c111', 'request_id': 'req-46e74855-d3f6-4303-a991-ca9a98215d07'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:54.534158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-46e74855-d3f6-4303-a991-ca9a98215d07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6810fa0c024f4e9002a7f519d2c111', 'request_id': 'req-46e74855-d3f6-4303-a991-ca9a98215d07'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:54.537206 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:54.560570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-46e74855-d3f6-4303-a991-ca9a98215d07 admin admin] The token cache is being invalidate because project 9b6810fa0c024f4e9002a7f519d2c111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:54.561583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-46e74855-d3f6-4303-a991-ca9a98215d07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:54.562817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-46e74855-d3f6-4303-a991-ca9a98215d07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:54.564679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1010/2029] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:54 2022] DELETE /identity/v3/projects/9b6810fa0c024f4e9002a7f519d2c111 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:54.570660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeaaca-93a2-4fe2-bfe0-cf2cda627dd5 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:54.571073 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeaaca-93a2-4fe2-bfe0-cf2cda627dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:54.571473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeaaca-93a2-4fe2-bfe0-cf2cda627dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:54.584282 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479f6c0-106f-4fdd-a281-cb002acf8b46 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:54.615007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-21eeaaca-93a2-4fe2-bfe0-cf2cda627dd5 None None] MFA Rules not processed for user `1eb4a1ce172e41c6967b003598247be1`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:54.616289 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-21eeaaca-93a2-4fe2-bfe0-cf2cda627dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:54.681324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03b6a3-b909-4375-9090-436822a1ebdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:54.682263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03b6a3-b909-4375-9090-436822a1ebdb service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:54.682690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03b6a3-b909-4375-9090-436822a1ebdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:54.683392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03b6a3-b909-4375-9090-436822a1ebdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:54.686073 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ef03b6a3-b909-4375-9090-436822a1ebdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:54.768302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03b6a3-b909-4375-9090-436822a1ebdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:54.770144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03b6a3-b909-4375-9090-436822a1ebdb service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:54.774730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1011/2030] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:54.774730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645c921-ccb8-4652-af9f-e762d4364020 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:54.774730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645c921-ccb8-4652-af9f-e762d4364020 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:54.774730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645c921-ccb8-4652-af9f-e762d4364020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:54.786708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1020/2031] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:54.788559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ee66d-945b-4133-8b7e-81c4bdd2d8a3 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:54.788945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ee66d-945b-4133-8b7e-81c4bdd2d8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:54.789849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ee66d-945b-4133-8b7e-81c4bdd2d8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:54.813684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8645c921-ccb8-4652-af9f-e762d4364020 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:54.815384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8645c921-ccb8-4652-af9f-e762d4364020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:54.837023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1012/2032] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:54.844671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645c921-ccb8-4652-af9f-e762d4364020 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:54.849569 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8645c921-ccb8-4652-af9f-e762d4364020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:54.860288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-ba9ee66d-945b-4133-8b7e-81c4bdd2d8a3 None None] MFA Rules not processed for user `3f93418e3e264a1f8597f789ee888b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:54.861903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ba9ee66d-945b-4133-8b7e-81c4bdd2d8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:54.869526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:55.000255 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1021/2033] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:55.002524 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7dd28-5837-4791-8792-fb834a4d72e4 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:55.002917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645c921-ccb8-4652-af9f-e762d4364020 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:55.004061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7dd28-5837-4791-8792-fb834a4d72e4 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:55.004484 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7dd28-5837-4791-8792-fb834a4d72e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:55.012633 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01538d4f-f2f3-4b19-889f-1425b2e6cd67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:55.014219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01538d4f-f2f3-4b19-889f-1425b2e6cd67 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:55.014616 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01538d4f-f2f3-4b19-889f-1425b2e6cd67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:55.015027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01538d4f-f2f3-4b19-889f-1425b2e6cd67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:55.020724 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-01538d4f-f2f3-4b19-889f-1425b2e6cd67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:55.054204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-e2d7dd28-5837-4791-8792-fb834a4d72e4 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:55.056497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e2d7dd28-5837-4791-8792-fb834a4d72e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:55.075678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1022/2034] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:55.077096 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7dd28-5837-4791-8792-fb834a4d72e4 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:55.078712 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e2d7dd28-5837-4791-8792-fb834a4d72e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:55.108584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7dd28-5837-4791-8792-fb834a4d72e4 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:55.114192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caeb7a3-558e-444d-84be-305437332f36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:55.114942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caeb7a3-558e-444d-84be-305437332f36 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:55.115591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caeb7a3-558e-444d-84be-305437332f36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:55.115934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caeb7a3-558e-444d-84be-305437332f36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:55.134595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caeb7a3-558e-444d-84be-305437332f36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:55.138260 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caeb7a3-558e-444d-84be-305437332f36 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:55.141007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01538d4f-f2f3-4b19-889f-1425b2e6cd67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:55.143914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01538d4f-f2f3-4b19-889f-1425b2e6cd67 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:55.152060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1023/2035] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:55.152635 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caeb7a3-558e-444d-84be-305437332f36 service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:55.159257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1013/2036] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:55.169895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caeb7a3-558e-444d-84be-305437332f36 service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:55.182706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fa1da-df6c-4d8e-a064-70f9358ae85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:55.183912 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fa1da-df6c-4d8e-a064-70f9358ae85f admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:55.184340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fa1da-df6c-4d8e-a064-70f9358ae85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:55.184834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fa1da-df6c-4d8e-a064-70f9358ae85f admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:55.185686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fa1da-df6c-4d8e-a064-70f9358ae85f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:17:55.186001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fa1da-df6c-4d8e-a064-70f9358ae85f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:55.188531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fa1da-df6c-4d8e-a064-70f9358ae85f admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:55.209479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01538d4f-f2f3-4b19-889f-1425b2e6cd67 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:55.211608 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1024/2037] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:17:55.225503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01538d4f-f2f3-4b19-889f-1425b2e6cd67 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:55.235410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367daf64-7e2a-43e3-a9c7-f01b81740cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:55.240034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367daf64-7e2a-43e3-a9c7-f01b81740cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:55.240733 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367daf64-7e2a-43e3-a9c7-f01b81740cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:55.241016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367daf64-7e2a-43e3-a9c7-f01b81740cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:55.243341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367daf64-7e2a-43e3-a9c7-f01b81740cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:55.245478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367daf64-7e2a-43e3-a9c7-f01b81740cc0 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:55.285404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1014/2038] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:55 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:55.293191 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fa1da-df6c-4d8e-a064-70f9358ae85f admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:55.309143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fa1da-df6c-4d8e-a064-70f9358ae85f admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:55.407541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4256c-95d6-4d75-8432-93305bc2ed83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:55.408126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4256c-95d6-4d75-8432-93305bc2ed83 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:55.408414 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4256c-95d6-4d75-8432-93305bc2ed83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:55.408693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4256c-95d6-4d75-8432-93305bc2ed83 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:55.409511 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4256c-95d6-4d75-8432-93305bc2ed83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:55.412855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4256c-95d6-4d75-8432-93305bc2ed83 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:55.459865 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1025/2039] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:55 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:55.466087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367daf64-7e2a-43e3-a9c7-f01b81740cc0 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:55.483662 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367daf64-7e2a-43e3-a9c7-f01b81740cc0 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:55.505234 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487dc2f-b1ed-4bf5-9919-9471273e4559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:55.505974 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487dc2f-b1ed-4bf5-9919-9471273e4559 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:55.506263 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487dc2f-b1ed-4bf5-9919-9471273e4559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:55.506573 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487dc2f-b1ed-4bf5-9919-9471273e4559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:55.508148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487dc2f-b1ed-4bf5-9919-9471273e4559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:55.511463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487dc2f-b1ed-4bf5-9919-9471273e4559 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:55.517099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:55.551792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1015/2040] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:55.557043 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4256c-95d6-4d75-8432-93305bc2ed83 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:55.569543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487dc2f-b1ed-4bf5-9919-9471273e4559 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:55.570511 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4256c-95d6-4d75-8432-93305bc2ed83 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:55.574067 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2487dc2f-b1ed-4bf5-9919-9471273e4559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:55.587608 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8c1ca-e271-4be5-b4f0-9bef22c576af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:55.589415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8c1ca-e271-4be5-b4f0-9bef22c576af admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:55.589700 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8c1ca-e271-4be5-b4f0-9bef22c576af admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:55.590829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8c1ca-e271-4be5-b4f0-9bef22c576af admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:55.595130 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8c1ca-e271-4be5-b4f0-9bef22c576af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:55.602747 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8c1ca-e271-4be5-b4f0-9bef22c576af admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:55.618043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487dc2f-b1ed-4bf5-9919-9471273e4559 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:55.632144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1026/2041] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:55.633591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8c1ca-e271-4be5-b4f0-9bef22c576af admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:55.636021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:55.636721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:55.637000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:55.637323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] PATH_INFO: `/v3/projects/8019caca03bd4cd7b440c3b15e5e2f6f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:55.640517 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8019caca03bd4cd7b440c3b15e5e2f6f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:55.642497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:55.660474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8c1ca-e271-4be5-b4f0-9bef22c576af admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:55.690948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052c352-0452-4a2a-8f9e-e2bfb04ff72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:55.690948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052c352-0452-4a2a-8f9e-e2bfb04ff72e admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:55.691420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052c352-0452-4a2a-8f9e-e2bfb04ff72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:55.691857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052c352-0452-4a2a-8f9e-e2bfb04ff72e admin admin] PATH_INFO: `/v3/projects/ac49dd585dc447da985de9c3f63aea1a/users/d60dc9146a0849cb9a0836d084732e8e/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:55.738123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052c352-0452-4a2a-8f9e-e2bfb04ff72e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac49dd585dc447da985de9c3f63aea1a, user_id=d60dc9146a0849cb9a0836d084732e8e, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:55.742179 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052c352-0452-4a2a-8f9e-e2bfb04ff72e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:55.784552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8019caca03bd4cd7b440c3b15e5e2f6f', 'request_id': 'req-2842cef4-54ab-4173-99d4-1ea192181f0b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:55.786734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1027/2042] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:55 2022] PUT /identity/v3/projects/ac49dd585dc447da985de9c3f63aea1a/users/d60dc9146a0849cb9a0836d084732e8e/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:55.788384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f04b0-3ffc-4ef6-b67f-f02acb05b8f1 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:55.788715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f04b0-3ffc-4ef6-b67f-f02acb05b8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:55.789002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f04b0-3ffc-4ef6-b67f-f02acb05b8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:55.807860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8019caca03bd4cd7b440c3b15e5e2f6f', 'request_id': 'req-2842cef4-54ab-4173-99d4-1ea192181f0b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:55.825045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-646f04b0-3ffc-4ef6-b67f-f02acb05b8f1 None None] MFA Rules not processed for user `d60dc9146a0849cb9a0836d084732e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:55.826369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-646f04b0-3ffc-4ef6-b67f-f02acb05b8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:55.827825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8019caca03bd4cd7b440c3b15e5e2f6f', 'request_id': 'req-2842cef4-54ab-4173-99d4-1ea192181f0b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:55.848708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:55.863022 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] The token cache is being invalidate because project 8019caca03bd4cd7b440c3b15e5e2f6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:55.863412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:55.865255 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:55.871912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1016/2043] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:55 2022] DELETE /identity/v3/projects/8019caca03bd4cd7b440c3b15e5e2f6f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:55.884103 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:55.891009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:55.927613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1028/2044] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:55.972764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f04b0-3ffc-4ef6-b67f-f02acb05b8f1 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:55.975128 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-646f04b0-3ffc-4ef6-b67f-f02acb05b8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:56.057746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842cef4-54ab-4173-99d4-1ea192181f0b admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:56.143413 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f04b0-3ffc-4ef6-b67f-f02acb05b8f1 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:56.156555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e6e73-a2cf-4845-99b9-4b3b9055ae9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:56.156555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e6e73-a2cf-4845-99b9-4b3b9055ae9d service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:56.156555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e6e73-a2cf-4845-99b9-4b3b9055ae9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:56.156555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e6e73-a2cf-4845-99b9-4b3b9055ae9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:56.179080 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e6e73-a2cf-4845-99b9-4b3b9055ae9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:56.182822 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e6e73-a2cf-4845-99b9-4b3b9055ae9d service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:56.204758 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1029/2045] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:56.222638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff91b85-7e77-4015-8655-4a0d819c12ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:56.225157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff91b85-7e77-4015-8655-4a0d819c12ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:56.225813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff91b85-7e77-4015-8655-4a0d819c12ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:56.226320 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff91b85-7e77-4015-8655-4a0d819c12ef admin admin] PATH_INFO: `/v3/users/ecd163d5d3944a168f5c063be0e592c4` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:56.232009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff91b85-7e77-4015-8655-4a0d819c12ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd163d5d3944a168f5c063be0e592c4)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:56.235086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff91b85-7e77-4015-8655-4a0d819c12ef admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:56.299561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:56.343985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-bff91b85-7e77-4015-8655-4a0d819c12ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd163d5d3944a168f5c063be0e592c4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:56.359057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-bff91b85-7e77-4015-8655-4a0d819c12ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd163d5d3944a168f5c063be0e592c4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:56.360368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-bff91b85-7e77-4015-8655-4a0d819c12ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd163d5d3944a168f5c063be0e592c4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:56.360837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-bff91b85-7e77-4015-8655-4a0d819c12ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd163d5d3944a168f5c063be0e592c4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:56.385974 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-bff91b85-7e77-4015-8655-4a0d819c12ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd163d5d3944a168f5c063be0e592c4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:56.403838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1017/2046] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:55 2022] DELETE /identity/v3/users/ecd163d5d3944a168f5c063be0e592c4 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:56.878834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e6e73-a2cf-4845-99b9-4b3b9055ae9d service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:56.895587 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e6e73-a2cf-4845-99b9-4b3b9055ae9d service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:56.981023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bf5ff-14cf-4be5-947b-82e030431db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:56.981794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bf5ff-14cf-4be5-947b-82e030431db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:56.982078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bf5ff-14cf-4be5-947b-82e030431db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:56.982354 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bf5ff-14cf-4be5-947b-82e030431db4 admin admin] PATH_INFO: `/v3/projects/f092c4f68a874b3dae48d921830c8cfc` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:56.983796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bf5ff-14cf-4be5-947b-82e030431db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f092c4f68a874b3dae48d921830c8cfc)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:56.985728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bf5ff-14cf-4be5-947b-82e030431db4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:57.037648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ab2bf5ff-14cf-4be5-947b-82e030431db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f092c4f68a874b3dae48d921830c8cfc', 'request_id': 'req-ab2bf5ff-14cf-4be5-947b-82e030431db4'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:57.045464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ab2bf5ff-14cf-4be5-947b-82e030431db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f092c4f68a874b3dae48d921830c8cfc', 'request_id': 'req-ab2bf5ff-14cf-4be5-947b-82e030431db4'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:57.051335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ab2bf5ff-14cf-4be5-947b-82e030431db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f092c4f68a874b3dae48d921830c8cfc', 'request_id': 'req-ab2bf5ff-14cf-4be5-947b-82e030431db4'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:57.053097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:57.065829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ab2bf5ff-14cf-4be5-947b-82e030431db4 admin admin] The token cache is being invalidate because project f092c4f68a874b3dae48d921830c8cfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:17:57.066123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ab2bf5ff-14cf-4be5-947b-82e030431db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:57.066771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ab2bf5ff-14cf-4be5-947b-82e030431db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:57.068326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1030/2047] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:17:56 2022] DELETE /identity/v3/projects/f092c4f68a874b3dae48d921830c8cfc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:57.074622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f80e1-600b-4a3e-b3ba-0ef0e0eec5dd None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:57.075371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f80e1-600b-4a3e-b3ba-0ef0e0eec5dd None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:57.079082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f80e1-600b-4a3e-b3ba-0ef0e0eec5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:57.108563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-fb8f80e1-600b-4a3e-b3ba-0ef0e0eec5dd None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:57.109902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-fb8f80e1-600b-4a3e-b3ba-0ef0e0eec5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:57.185650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1018/2048] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:57.193096 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195962f-a177-43c6-8651-db31f1c8adb8 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:57.193385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195962f-a177-43c6-8651-db31f1c8adb8 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:57.193666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195962f-a177-43c6-8651-db31f1c8adb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:57.216130 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a195962f-a177-43c6-8651-db31f1c8adb8 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:57.217357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a195962f-a177-43c6-8651-db31f1c8adb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:57.226101 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1031/2049] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:57.230891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f80e1-600b-4a3e-b3ba-0ef0e0eec5dd None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:57.244973 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f80e1-600b-4a3e-b3ba-0ef0e0eec5dd None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:57.250018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0cc87-0b52-4630-a8d8-03cff40974ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:57.250650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0cc87-0b52-4630-a8d8-03cff40974ea admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:57.250952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0cc87-0b52-4630-a8d8-03cff40974ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:57.251281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0cc87-0b52-4630-a8d8-03cff40974ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:57.251991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0cc87-0b52-4630-a8d8-03cff40974ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:17:57.252339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0cc87-0b52-4630-a8d8-03cff40974ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:57.254020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0cc87-0b52-4630-a8d8-03cff40974ea admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:57.267921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1019/2050] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:17:57.272440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195962f-a177-43c6-8651-db31f1c8adb8 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:57.283303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195962f-a177-43c6-8651-db31f1c8adb8 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:57.288130 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58d3a8-c118-45f6-86e6-5c1e3d52f7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:57.288743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58d3a8-c118-45f6-86e6-5c1e3d52f7ba admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:57.289025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58d3a8-c118-45f6-86e6-5c1e3d52f7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:57.289297 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58d3a8-c118-45f6-86e6-5c1e3d52f7ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:57.289954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58d3a8-c118-45f6-86e6-5c1e3d52f7ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:57.292415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58d3a8-c118-45f6-86e6-5c1e3d52f7ba admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:57.326344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1032/2051] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:57 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:57.330583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0cc87-0b52-4630-a8d8-03cff40974ea admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:57.346482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0cc87-0b52-4630-a8d8-03cff40974ea admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:57.418649 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3bdae-5b11-448d-ac2e-b0abc9524cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:57.419472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3bdae-5b11-448d-ac2e-b0abc9524cda admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:57.419748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3bdae-5b11-448d-ac2e-b0abc9524cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:57.419984 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3bdae-5b11-448d-ac2e-b0abc9524cda admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:57.420629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3bdae-5b11-448d-ac2e-b0abc9524cda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:57.422618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3bdae-5b11-448d-ac2e-b0abc9524cda admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:57.447657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1020/2052] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:57 2022] POST /identity/v3/users => generated 291 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:57.452596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58d3a8-c118-45f6-86e6-5c1e3d52f7ba admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:57.473060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58d3a8-c118-45f6-86e6-5c1e3d52f7ba admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:57.479241 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666a484-2792-4690-be7b-94679ba18c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:57.480003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666a484-2792-4690-be7b-94679ba18c6e admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:57.480380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666a484-2792-4690-be7b-94679ba18c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:57.480660 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666a484-2792-4690-be7b-94679ba18c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:57.481322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666a484-2792-4690-be7b-94679ba18c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:57.483307 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666a484-2792-4690-be7b-94679ba18c6e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:57.495653 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1033/2053] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:57.499706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3bdae-5b11-448d-ac2e-b0abc9524cda admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:57.510918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3bdae-5b11-448d-ac2e-b0abc9524cda admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:57.516653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbca7de-f34b-4cb7-a818-91db1adbdce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:57.517293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbca7de-f34b-4cb7-a818-91db1adbdce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:57.517571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbca7de-f34b-4cb7-a818-91db1adbdce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:57.517844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbca7de-f34b-4cb7-a818-91db1adbdce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:57.518508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbca7de-f34b-4cb7-a818-91db1adbdce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:57.520228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbca7de-f34b-4cb7-a818-91db1adbdce8 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:57.531851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1021/2054] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:57.536022 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666a484-2792-4690-be7b-94679ba18c6e admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:57.547931 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666a484-2792-4690-be7b-94679ba18c6e admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:57.552961 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ea1f1-b0b1-4ce7-b23d-42759d7c65cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:57.553621 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ea1f1-b0b1-4ce7-b23d-42759d7c65cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:57.553898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ea1f1-b0b1-4ce7-b23d-42759d7c65cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:57.554176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ea1f1-b0b1-4ce7-b23d-42759d7c65cd admin admin] PATH_INFO: `/v3/projects/4ef8310b91934f3a90bbaa2af3604766/users/83934c7d4c844691ba9ab61c6f6f98c1/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:57.573650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ea1f1-b0b1-4ce7-b23d-42759d7c65cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef8310b91934f3a90bbaa2af3604766, user_id=83934c7d4c844691ba9ab61c6f6f98c1, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:57.576395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ea1f1-b0b1-4ce7-b23d-42759d7c65cd admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:57.590480 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1034/2055] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:57 2022] PUT /identity/v3/projects/4ef8310b91934f3a90bbaa2af3604766/users/83934c7d4c844691ba9ab61c6f6f98c1/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:17:57.603650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62762d0-ea1b-48f9-9bf4-11484075a7ab None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:57.603928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62762d0-ea1b-48f9-9bf4-11484075a7ab None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:57.606502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62762d0-ea1b-48f9-9bf4-11484075a7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:57.629288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:57.638495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f62762d0-ea1b-48f9-9bf4-11484075a7ab None None] MFA Rules not processed for user `83934c7d4c844691ba9ab61c6f6f98c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:57.640322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f62762d0-ea1b-48f9-9bf4-11484075a7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:57.710353 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1022/2056] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:17:57.715600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ea1f1-b0b1-4ce7-b23d-42759d7c65cd admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:57.796069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ea1f1-b0b1-4ce7-b23d-42759d7c65cd admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:57.801032 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3e00b-b407-49fe-88f7-2db5cd62e38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:57.801642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3e00b-b407-49fe-88f7-2db5cd62e38d admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:57.801933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3e00b-b407-49fe-88f7-2db5cd62e38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:57.802216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3e00b-b407-49fe-88f7-2db5cd62e38d admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:57.802877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca3e00b-b407-49fe-88f7-2db5cd62e38d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:57.805023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca3e00b-b407-49fe-88f7-2db5cd62e38d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:57.834266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1035/2057] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:17:57 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:57.839628 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62762d0-ea1b-48f9-9bf4-11484075a7ab None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:57.851889 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62762d0-ea1b-48f9-9bf4-11484075a7ab None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:57.914993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa6bdc-ef22-4628-8645-d0aec452d481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:57.915457 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa6bdc-ef22-4628-8645-d0aec452d481 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:57.915740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa6bdc-ef22-4628-8645-d0aec452d481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:57.916008 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa6bdc-ef22-4628-8645-d0aec452d481 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:57.916673 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa6bdc-ef22-4628-8645-d0aec452d481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:57.918652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa6bdc-ef22-4628-8645-d0aec452d481 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:57.948269 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1023/2058] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:17:57 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:17:57.953789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3e00b-b407-49fe-88f7-2db5cd62e38d admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:57.971267 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3e00b-b407-49fe-88f7-2db5cd62e38d admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:57.976154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11deda9d-b8b9-4258-8c08-b7de0f45a218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:57.976868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deda9d-b8b9-4258-8c08-b7de0f45a218 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:57.977224 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deda9d-b8b9-4258-8c08-b7de0f45a218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:57.977533 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deda9d-b8b9-4258-8c08-b7de0f45a218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:57.978184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11deda9d-b8b9-4258-8c08-b7de0f45a218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:57.979842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11deda9d-b8b9-4258-8c08-b7de0f45a218 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:57.992183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1036/2059] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:17:57.996150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa6bdc-ef22-4628-8645-d0aec452d481 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:58.006609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa6bdc-ef22-4628-8645-d0aec452d481 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:58.012047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37fd9a-5d8e-47e7-8f93-6b3734f47ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:58.012725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37fd9a-5d8e-47e7-8f93-6b3734f47ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:58.013000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37fd9a-5d8e-47e7-8f93-6b3734f47ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:58.013279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37fd9a-5d8e-47e7-8f93-6b3734f47ea4 admin admin] PATH_INFO: `/v3/projects/a8e26e9ff55f4eb6aae214826b9bd2cc/users/001bcef0f926420d8ee657d05c575fb0/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:58.030883 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37fd9a-5d8e-47e7-8f93-6b3734f47ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8e26e9ff55f4eb6aae214826b9bd2cc, user_id=001bcef0f926420d8ee657d05c575fb0, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:58.033653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37fd9a-5d8e-47e7-8f93-6b3734f47ea4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:58.058612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1024/2060] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:57 2022] PUT /identity/v3/projects/a8e26e9ff55f4eb6aae214826b9bd2cc/users/001bcef0f926420d8ee657d05c575fb0/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:58.061072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11deda9d-b8b9-4258-8c08-b7de0f45a218 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:58.077424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11deda9d-b8b9-4258-8c08-b7de0f45a218 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:58.139183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c9de7-e8f5-446e-af2f-5c92cd1f2e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:58.139876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c9de7-e8f5-446e-af2f-5c92cd1f2e69 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:58.140163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c9de7-e8f5-446e-af2f-5c92cd1f2e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:58.140441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c9de7-e8f5-446e-af2f-5c92cd1f2e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:58.141094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c9de7-e8f5-446e-af2f-5c92cd1f2e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:58.142711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c9de7-e8f5-446e-af2f-5c92cd1f2e69 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:58.145198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:58.157129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1037/2061] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:17:58.160571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37fd9a-5d8e-47e7-8f93-6b3734f47ea4 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:58.172209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37fd9a-5d8e-47e7-8f93-6b3734f47ea4 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:58.177136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cbde6-10e7-439b-ada9-9a8e10612f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:58.177905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cbde6-10e7-439b-ada9-9a8e10612f95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:58.178247 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cbde6-10e7-439b-ada9-9a8e10612f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:58.178585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cbde6-10e7-439b-ada9-9a8e10612f95 admin admin] PATH_INFO: `/v3/projects/a8e26e9ff55f4eb6aae214826b9bd2cc/users/001bcef0f926420d8ee657d05c575fb0/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:58.180878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563cbde6-10e7-439b-ada9-9a8e10612f95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8e26e9ff55f4eb6aae214826b9bd2cc, user_id=001bcef0f926420d8ee657d05c575fb0, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:58.184422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563cbde6-10e7-439b-ada9-9a8e10612f95 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:58.198506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1025/2062] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:17:58 2022] PUT /identity/v3/projects/a8e26e9ff55f4eb6aae214826b9bd2cc/users/001bcef0f926420d8ee657d05c575fb0/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:58.210015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e2bea-a606-46b2-b784-06db05dd3dd5 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:58.210350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e2bea-a606-46b2-b784-06db05dd3dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:58.211220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e2bea-a606-46b2-b784-06db05dd3dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:58.213407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d4795-38a9-4a6f-ab6c-52aff0f17a84 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:58.213699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d4795-38a9-4a6f-ab6c-52aff0f17a84 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:58.213981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d4795-38a9-4a6f-ab6c-52aff0f17a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:58.234058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-c48d4795-38a9-4a6f-ab6c-52aff0f17a84 None None] MFA Rules not processed for user `3f93418e3e264a1f8597f789ee888b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:58.235431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c48d4795-38a9-4a6f-ab6c-52aff0f17a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:58.247718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-1a7e2bea-a606-46b2-b784-06db05dd3dd5 None None] MFA Rules not processed for user `001bcef0f926420d8ee657d05c575fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:58.249442 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1a7e2bea-a606-46b2-b784-06db05dd3dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:58.311975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:58.326411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1038/2063] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:58.371248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1026/2064] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:58.390090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d4795-38a9-4a6f-ab6c-52aff0f17a84 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:58.391050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c48d4795-38a9-4a6f-ab6c-52aff0f17a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:58.468467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d4795-38a9-4a6f-ab6c-52aff0f17a84 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:58.472853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba360e-8f42-40f5-b34d-32f95e6d6fd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:58.473462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba360e-8f42-40f5-b34d-32f95e6d6fd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:58.473737 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba360e-8f42-40f5-b34d-32f95e6d6fd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:58.474008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba360e-8f42-40f5-b34d-32f95e6d6fd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:58.487234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba360e-8f42-40f5-b34d-32f95e6d6fd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:58.488889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba360e-8f42-40f5-b34d-32f95e6d6fd7 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:58.503393 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1039/2065] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:17:59.105541 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e2bea-a606-46b2-b784-06db05dd3dd5 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:59.200536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e2bea-a606-46b2-b784-06db05dd3dd5 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:59.206198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:59.207134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:59.207532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:59.207962 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] PATH_INFO: `/v3/users/83934c7d4c844691ba9ab61c6f6f98c1` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:59.209352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] RBAC: Authorizing `identity:delete_user(user_id=83934c7d4c844691ba9ab61c6f6f98c1)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:59.211526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:59.317882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83934c7d4c844691ba9ab61c6f6f98c1'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:59.337162 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83934c7d4c844691ba9ab61c6f6f98c1'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:59.338875 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83934c7d4c844691ba9ab61c6f6f98c1'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:59.339813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83934c7d4c844691ba9ab61c6f6f98c1'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:59.375458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83934c7d4c844691ba9ab61c6f6f98c1'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:17:59.394699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1027/2066] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:17:59 2022] DELETE /identity/v3/users/83934c7d4c844691ba9ab61c6f6f98c1 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:17:59.399312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd3f10-edd1-4917-bf9d-370302d440dc None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:59.399819 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd3f10-edd1-4917-bf9d-370302d440dc None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:59.399918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd3f10-edd1-4917-bf9d-370302d440dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:59.454214 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-62fd3f10-edd1-4917-bf9d-370302d440dc None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:17:59.455895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-62fd3f10-edd1-4917-bf9d-370302d440dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:59.544051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1040/2067] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:17:59.556124 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:17:59.557869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:17:59.663576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2882362-8d65-45a7-84f7-9eca5e1af16a admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:17:59.678695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e04a1-9efa-4bde-acea-35d2eb6c9613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:17:59.678695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e04a1-9efa-4bde-acea-35d2eb6c9613 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:17:59.678695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e04a1-9efa-4bde-acea-35d2eb6c9613 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:17:59.678695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e04a1-9efa-4bde-acea-35d2eb6c9613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:17:59.691480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e04a1-9efa-4bde-acea-35d2eb6c9613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:17:59.693578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e04a1-9efa-4bde-acea-35d2eb6c9613 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:17:59.702410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:17:59.712552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1028/2068] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:17:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:00.062585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd3f10-edd1-4917-bf9d-370302d440dc None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:00.064953 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-62fd3f10-edd1-4917-bf9d-370302d440dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:00.083075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd3f10-edd1-4917-bf9d-370302d440dc None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:00.089677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:00.090349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:00.090633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:00.090912 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] PATH_INFO: `/v3/projects/4ef8310b91934f3a90bbaa2af3604766` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:00.101388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ef8310b91934f3a90bbaa2af3604766)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:00.103918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:00.174278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:00.201519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ef8310b91934f3a90bbaa2af3604766', 'request_id': 'req-424633aa-64f3-442c-894c-3d1ab952c7db'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:00.214707 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ef8310b91934f3a90bbaa2af3604766', 'request_id': 'req-424633aa-64f3-442c-894c-3d1ab952c7db'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:00.232197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ef8310b91934f3a90bbaa2af3604766', 'request_id': 'req-424633aa-64f3-442c-894c-3d1ab952c7db'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:00.277486 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] The token cache is being invalidate because project 4ef8310b91934f3a90bbaa2af3604766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:00.277830 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:00.280474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:00.285778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1041/2069] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:00 2022] DELETE /identity/v3/projects/4ef8310b91934f3a90bbaa2af3604766 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:00.288246 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e04a1-9efa-4bde-acea-35d2eb6c9613 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:00.309279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e04a1-9efa-4bde-acea-35d2eb6c9613 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:00.458107 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:00.459063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:00.459326 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:00.459618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] PATH_INFO: `/v3/users/001bcef0f926420d8ee657d05c575fb0` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:00.461444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] RBAC: Authorizing `identity:delete_user(user_id=001bcef0f926420d8ee657d05c575fb0)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:00.463748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:00.522443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '001bcef0f926420d8ee657d05c575fb0'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:00.532230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '001bcef0f926420d8ee657d05c575fb0'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:00.533446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '001bcef0f926420d8ee657d05c575fb0'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:00.534585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '001bcef0f926420d8ee657d05c575fb0'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:00.549972 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '001bcef0f926420d8ee657d05c575fb0'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:00.567134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1029/2070] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:00 2022] DELETE /identity/v3/users/001bcef0f926420d8ee657d05c575fb0 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:00.833721 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:00.835459 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:00.856422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424633aa-64f3-442c-894c-3d1ab952c7db admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:00.920560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac9f15-216f-4237-9414-f8439b3cd766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:00.921267 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac9f15-216f-4237-9414-f8439b3cd766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:00.921614 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac9f15-216f-4237-9414-f8439b3cd766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:00.921966 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac9f15-216f-4237-9414-f8439b3cd766 admin admin] PATH_INFO: `/v3/users/d60dc9146a0849cb9a0836d084732e8e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:00.923275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac9f15-216f-4237-9414-f8439b3cd766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d60dc9146a0849cb9a0836d084732e8e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:00.925240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac9f15-216f-4237-9414-f8439b3cd766 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:00.931434 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:00.933120 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:00.956018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4622bf-8193-42bf-8bc8-1a99f1b3865d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:00.961293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787dbc5-65cb-4181-b1e8-39ad737996e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:00.962077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787dbc5-65cb-4181-b1e8-39ad737996e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:00.962655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787dbc5-65cb-4181-b1e8-39ad737996e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:00.962855 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787dbc5-65cb-4181-b1e8-39ad737996e5 admin admin] PATH_INFO: `/v3/projects/a8e26e9ff55f4eb6aae214826b9bd2cc` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:00.964688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787dbc5-65cb-4181-b1e8-39ad737996e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8e26e9ff55f4eb6aae214826b9bd2cc)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:00.966733 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787dbc5-65cb-4181-b1e8-39ad737996e5 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:00.999974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1aac9f15-216f-4237-9414-f8439b3cd766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd60dc9146a0849cb9a0836d084732e8e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.013681 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1aac9f15-216f-4237-9414-f8439b3cd766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd60dc9146a0849cb9a0836d084732e8e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.020206 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e787dbc5-65cb-4181-b1e8-39ad737996e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8e26e9ff55f4eb6aae214826b9bd2cc', 'request_id': 'req-e787dbc5-65cb-4181-b1e8-39ad737996e5'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.021394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1aac9f15-216f-4237-9414-f8439b3cd766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd60dc9146a0849cb9a0836d084732e8e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.022101 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1aac9f15-216f-4237-9414-f8439b3cd766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd60dc9146a0849cb9a0836d084732e8e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.022728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-1aac9f15-216f-4237-9414-f8439b3cd766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd60dc9146a0849cb9a0836d084732e8e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.029256 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e787dbc5-65cb-4181-b1e8-39ad737996e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8e26e9ff55f4eb6aae214826b9bd2cc', 'request_id': 'req-e787dbc5-65cb-4181-b1e8-39ad737996e5'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.035611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e787dbc5-65cb-4181-b1e8-39ad737996e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8e26e9ff55f4eb6aae214826b9bd2cc', 'request_id': 'req-e787dbc5-65cb-4181-b1e8-39ad737996e5'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.037877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1042/2071] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:00 2022] DELETE /identity/v3/users/d60dc9146a0849cb9a0836d084732e8e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:01.042563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150bef1-aa83-4ced-abbc-bbf8ded63fcb None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.042906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150bef1-aa83-4ced-abbc-bbf8ded63fcb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.043453 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150bef1-aa83-4ced-abbc-bbf8ded63fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.048822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e787dbc5-65cb-4181-b1e8-39ad737996e5 admin admin] The token cache is being invalidate because project a8e26e9ff55f4eb6aae214826b9bd2cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:01.049110 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e787dbc5-65cb-4181-b1e8-39ad737996e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.049746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e787dbc5-65cb-4181-b1e8-39ad737996e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.051268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1030/2072] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:00 2022] DELETE /identity/v3/projects/a8e26e9ff55f4eb6aae214826b9bd2cc => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:01.056146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3347ae-f87b-4307-85af-880fe02a89c8 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.056436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3347ae-f87b-4307-85af-880fe02a89c8 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.056715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3347ae-f87b-4307-85af-880fe02a89c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.063980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-7150bef1-aa83-4ced-abbc-bbf8ded63fcb None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:01.065097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:01.065288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7150bef1-aa83-4ced-abbc-bbf8ded63fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:01.082814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-4b3347ae-f87b-4307-85af-880fe02a89c8 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:01.084178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-4b3347ae-f87b-4307-85af-880fe02a89c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:01.115197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:01.134954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1043/2073] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:01.145411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150bef1-aa83-4ced-abbc-bbf8ded63fcb None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:01.146842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7150bef1-aa83-4ced-abbc-bbf8ded63fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:01.153979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1031/2074] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:01.162185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8e6ed-158f-4714-b1af-5a883e2b90dd None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.162469 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8e6ed-158f-4714-b1af-5a883e2b90dd None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.162746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8e6ed-158f-4714-b1af-5a883e2b90dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.183175 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-aaf8e6ed-158f-4714-b1af-5a883e2b90dd None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:01.184427 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-aaf8e6ed-158f-4714-b1af-5a883e2b90dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:01.192935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1032/2075] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:01.197847 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8e6ed-158f-4714-b1af-5a883e2b90dd None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:01.207707 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8e6ed-158f-4714-b1af-5a883e2b90dd None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:01.214595 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1401f8-30ee-4179-818f-538d70e4c75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:01.215100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1401f8-30ee-4179-818f-538d70e4c75e admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.215404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1401f8-30ee-4179-818f-538d70e4c75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.215674 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1401f8-30ee-4179-818f-538d70e4c75e admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.216331 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1401f8-30ee-4179-818f-538d70e4c75e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:18:01.216633 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1401f8-30ee-4179-818f-538d70e4c75e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:01.218252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1401f8-30ee-4179-818f-538d70e4c75e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:01.226175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150bef1-aa83-4ced-abbc-bbf8ded63fcb None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:01.230930 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1033/2076] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:18:01.231093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02180d7d-8986-4734-91b6-920f1135579e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:01.231864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02180d7d-8986-4734-91b6-920f1135579e service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.232333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02180d7d-8986-4734-91b6-920f1135579e service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.233044 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02180d7d-8986-4734-91b6-920f1135579e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.235444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1401f8-30ee-4179-818f-538d70e4c75e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:01.246641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02180d7d-8986-4734-91b6-920f1135579e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:01.246963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1401f8-30ee-4179-818f-538d70e4c75e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:01.248139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02180d7d-8986-4734-91b6-920f1135579e service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:01.252333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d4cab-7b08-4e88-815c-05f246bd87cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:01.253084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d4cab-7b08-4e88-815c-05f246bd87cf admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.253468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d4cab-7b08-4e88-815c-05f246bd87cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.253913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d4cab-7b08-4e88-815c-05f246bd87cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.254738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d4cab-7b08-4e88-815c-05f246bd87cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:01.256928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d4cab-7b08-4e88-815c-05f246bd87cf admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:01.259371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1044/2077] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:18:01.294628 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1034/2078] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:01 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:01.296062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02180d7d-8986-4734-91b6-920f1135579e service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:01.306858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02180d7d-8986-4734-91b6-920f1135579e service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:01.382640 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a4653-8f9f-4767-a61f-999fc34de5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:01.383447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a4653-8f9f-4767-a61f-999fc34de5f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.383829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a4653-8f9f-4767-a61f-999fc34de5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.384040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a4653-8f9f-4767-a61f-999fc34de5f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.384700 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a4653-8f9f-4767-a61f-999fc34de5f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:01.386701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a4653-8f9f-4767-a61f-999fc34de5f5 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:01.414134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1045/2079] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:01 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:01.417755 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d4cab-7b08-4e88-815c-05f246bd87cf admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:01.432262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d4cab-7b08-4e88-815c-05f246bd87cf admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:01.438083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead63e49-c876-4d49-b25b-5f56aba32040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:01.438724 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead63e49-c876-4d49-b25b-5f56aba32040 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.439005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead63e49-c876-4d49-b25b-5f56aba32040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.439362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead63e49-c876-4d49-b25b-5f56aba32040 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.440141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead63e49-c876-4d49-b25b-5f56aba32040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:01.441916 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead63e49-c876-4d49-b25b-5f56aba32040 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:01.459502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1035/2080] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:01.463906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a4653-8f9f-4767-a61f-999fc34de5f5 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:01.481619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a4653-8f9f-4767-a61f-999fc34de5f5 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:01.496748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15e0f8-b5d0-4fb0-a88f-93d83a491349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:01.497738 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15e0f8-b5d0-4fb0-a88f-93d83a491349 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.498421 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15e0f8-b5d0-4fb0-a88f-93d83a491349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.498817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15e0f8-b5d0-4fb0-a88f-93d83a491349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.500497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15e0f8-b5d0-4fb0-a88f-93d83a491349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:01.503123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15e0f8-b5d0-4fb0-a88f-93d83a491349 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:01.523470 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1046/2081] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:01.524670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead63e49-c876-4d49-b25b-5f56aba32040 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:01.537870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead63e49-c876-4d49-b25b-5f56aba32040 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:01.544881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b3e13-919e-4899-9140-c8afa1c2636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:01.545644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b3e13-919e-4899-9140-c8afa1c2636f admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.546001 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b3e13-919e-4899-9140-c8afa1c2636f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.546353 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b3e13-919e-4899-9140-c8afa1c2636f admin admin] PATH_INFO: `/v3/projects/7a1b904157204964b4c04d0772627052/users/d6ae71f1deb947379fa12f0c8f4ee02f/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.564340 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b3e13-919e-4899-9140-c8afa1c2636f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1b904157204964b4c04d0772627052, user_id=d6ae71f1deb947379fa12f0c8f4ee02f, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:01.567153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b3e13-919e-4899-9140-c8afa1c2636f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:01.582451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15e0f8-b5d0-4fb0-a88f-93d83a491349 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:01.584674 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bd15e0f8-b5d0-4fb0-a88f-93d83a491349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:01.587891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1036/2082] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:01 2022] PUT /identity/v3/projects/7a1b904157204964b4c04d0772627052/users/d6ae71f1deb947379fa12f0c8f4ee02f/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:01.600038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ffb94-40d7-4106-a545-a3488d68329a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.600951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ffb94-40d7-4106-a545-a3488d68329a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.602043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ffb94-40d7-4106-a545-a3488d68329a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.639872 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8e3ffb94-40d7-4106-a545-a3488d68329a None None] MFA Rules not processed for user `d6ae71f1deb947379fa12f0c8f4ee02f`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:01.641098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8e3ffb94-40d7-4106-a545-a3488d68329a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:01.664264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15e0f8-b5d0-4fb0-a88f-93d83a491349 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:01.698434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:01.730703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1037/2083] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:01.734889 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ffb94-40d7-4106-a545-a3488d68329a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:01.736032 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8e3ffb94-40d7-4106-a545-a3488d68329a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:01.738985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966531d8-222e-4215-b4a4-181fad253c5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:01.740001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966531d8-222e-4215-b4a4-181fad253c5f service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.740483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966531d8-222e-4215-b4a4-181fad253c5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.741006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966531d8-222e-4215-b4a4-181fad253c5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.745076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-966531d8-222e-4215-b4a4-181fad253c5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:01.759440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:01.820225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ffb94-40d7-4106-a545-a3488d68329a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:01.820508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966531d8-222e-4215-b4a4-181fad253c5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:01.821981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966531d8-222e-4215-b4a4-181fad253c5f service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:01.825837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef39dd-47e9-477e-962c-67dab86f247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:01.826718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef39dd-47e9-477e-962c-67dab86f247f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.827004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef39dd-47e9-477e-962c-67dab86f247f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.827304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef39dd-47e9-477e-962c-67dab86f247f admin admin] PATH_INFO: `/v3/projects/ac49dd585dc447da985de9c3f63aea1a` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.828891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef39dd-47e9-477e-962c-67dab86f247f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac49dd585dc447da985de9c3f63aea1a)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:01.831880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef39dd-47e9-477e-962c-67dab86f247f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:01.833213 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1047/2084] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:18:01.833786 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966531d8-222e-4215-b4a4-181fad253c5f service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:01.836534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-966531d8-222e-4215-b4a4-181fad253c5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:01.911757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ccef39dd-47e9-477e-962c-67dab86f247f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac49dd585dc447da985de9c3f63aea1a', 'request_id': 'req-ccef39dd-47e9-477e-962c-67dab86f247f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.926099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ccef39dd-47e9-477e-962c-67dab86f247f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac49dd585dc447da985de9c3f63aea1a', 'request_id': 'req-ccef39dd-47e9-477e-962c-67dab86f247f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.934183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ccef39dd-47e9-477e-962c-67dab86f247f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac49dd585dc447da985de9c3f63aea1a', 'request_id': 'req-ccef39dd-47e9-477e-962c-67dab86f247f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.946731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ccef39dd-47e9-477e-962c-67dab86f247f admin admin] The token cache is being invalidate because project ac49dd585dc447da985de9c3f63aea1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:01.947021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ccef39dd-47e9-477e-962c-67dab86f247f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.947725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ccef39dd-47e9-477e-962c-67dab86f247f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:01.949318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1038/2085] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:01 2022] DELETE /identity/v3/projects/ac49dd585dc447da985de9c3f63aea1a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:01.954643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5151c4c-20c8-45ed-b05d-c76afb052a42 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:01.954924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5151c4c-20c8-45ed-b05d-c76afb052a42 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:01.955227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5151c4c-20c8-45ed-b05d-c76afb052a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:01.979200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966531d8-222e-4215-b4a4-181fad253c5f service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:02.001221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-a5151c4c-20c8-45ed-b05d-c76afb052a42 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:02.003056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a5151c4c-20c8-45ed-b05d-c76afb052a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:02.059401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51974481-c61a-4cfd-abc8-a33369cf36cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f82adce75b44c6baf85b6b96dd0728', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:02.060029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51974481-c61a-4cfd-abc8-a33369cf36cb service glance] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:02.060343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51974481-c61a-4cfd-abc8-a33369cf36cb service glance] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:02.060800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51974481-c61a-4cfd-abc8-a33369cf36cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:02.063218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-51974481-c61a-4cfd-abc8-a33369cf36cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:02.107975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1039/2086] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:02.110117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:02.111937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecab3a-a1c8-44dc-be7e-28d39f92d3f0 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:02.112217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecab3a-a1c8-44dc-be7e-28d39f92d3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:02.112663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecab3a-a1c8-44dc-be7e-28d39f92d3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:02.133616 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-98ecab3a-a1c8-44dc-be7e-28d39f92d3f0 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:02.134848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-98ecab3a-a1c8-44dc-be7e-28d39f92d3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:02.144458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1040/2087] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:02.149495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecab3a-a1c8-44dc-be7e-28d39f92d3f0 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:02.153252 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51974481-c61a-4cfd-abc8-a33369cf36cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:02.154723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51974481-c61a-4cfd-abc8-a33369cf36cb service glance] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:02.159812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecab3a-a1c8-44dc-be7e-28d39f92d3f0 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:02.165413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b46009-d2d8-48dc-862b-0a511dbdaa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:02.166027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b46009-d2d8-48dc-862b-0a511dbdaa12 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:02.166309 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b46009-d2d8-48dc-862b-0a511dbdaa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:02.166580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b46009-d2d8-48dc-862b-0a511dbdaa12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:02.167356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b46009-d2d8-48dc-862b-0a511dbdaa12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:18:02.167650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b46009-d2d8-48dc-862b-0a511dbdaa12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:02.169546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1048/2088] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:02.170083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b46009-d2d8-48dc-862b-0a511dbdaa12 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:02.202723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1041/2089] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:18:02.204094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51974481-c61a-4cfd-abc8-a33369cf36cb service glance] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:02.220666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51974481-c61a-4cfd-abc8-a33369cf36cb service glance] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:02.225834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2310cd-d9cc-4bc3-a721-1a42a7cd431c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:02.227038 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2310cd-d9cc-4bc3-a721-1a42a7cd431c admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:02.227413 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2310cd-d9cc-4bc3-a721-1a42a7cd431c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:02.227817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2310cd-d9cc-4bc3-a721-1a42a7cd431c admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:02.228530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2310cd-d9cc-4bc3-a721-1a42a7cd431c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:02.230511 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2310cd-d9cc-4bc3-a721-1a42a7cd431c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:02.269057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1049/2090] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:02 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:02.278971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b46009-d2d8-48dc-862b-0a511dbdaa12 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:02.302562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b46009-d2d8-48dc-862b-0a511dbdaa12 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:02.333762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:02.381169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7faad-0c93-46c4-a69e-fbeae4d103ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:02.381783 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7faad-0c93-46c4-a69e-fbeae4d103ab admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:02.382597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7faad-0c93-46c4-a69e-fbeae4d103ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:02.383000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7faad-0c93-46c4-a69e-fbeae4d103ab admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:02.383550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7faad-0c93-46c4-a69e-fbeae4d103ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:02.385524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7faad-0c93-46c4-a69e-fbeae4d103ab admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:02.413621 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1042/2091] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:02 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:02.418397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2310cd-d9cc-4bc3-a721-1a42a7cd431c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:02.439968 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2310cd-d9cc-4bc3-a721-1a42a7cd431c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:02.446211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ee09a-e395-4dc2-be51-e8c3fd2c870b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:02.446858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ee09a-e395-4dc2-be51-e8c3fd2c870b admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:02.447167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ee09a-e395-4dc2-be51-e8c3fd2c870b admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:02.447451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ee09a-e395-4dc2-be51-e8c3fd2c870b admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:02.448105 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ee09a-e395-4dc2-be51-e8c3fd2c870b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:02.450067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ee09a-e395-4dc2-be51-e8c3fd2c870b admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:02.472412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1050/2092] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:02.476988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7faad-0c93-46c4-a69e-fbeae4d103ab admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:02.489847 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7faad-0c93-46c4-a69e-fbeae4d103ab admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:02.494874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c38a7-b348-4e61-8321-20e26e51af40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:02.495686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c38a7-b348-4e61-8321-20e26e51af40 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:02.495901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c38a7-b348-4e61-8321-20e26e51af40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:02.496262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c38a7-b348-4e61-8321-20e26e51af40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:02.496982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c38a7-b348-4e61-8321-20e26e51af40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:02.498711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c38a7-b348-4e61-8321-20e26e51af40 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:02.512623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1043/2093] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:02.517340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ee09a-e395-4dc2-be51-e8c3fd2c870b admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:02.529108 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ee09a-e395-4dc2-be51-e8c3fd2c870b admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:02.534717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50421d-b55c-4334-8fd7-5208c5d346a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:02.535531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50421d-b55c-4334-8fd7-5208c5d346a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:02.535883 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50421d-b55c-4334-8fd7-5208c5d346a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:02.536229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50421d-b55c-4334-8fd7-5208c5d346a2 admin admin] PATH_INFO: `/v3/projects/3d9e7f5459a3498a917d83a169e4eec6/users/563dda8b2f42478eb5c903230bc9fa7c/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:02.554318 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50421d-b55c-4334-8fd7-5208c5d346a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9e7f5459a3498a917d83a169e4eec6, user_id=563dda8b2f42478eb5c903230bc9fa7c, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:02.559969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50421d-b55c-4334-8fd7-5208c5d346a2 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:02.582094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1051/2094] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:02 2022] PUT /identity/v3/projects/3d9e7f5459a3498a917d83a169e4eec6/users/563dda8b2f42478eb5c903230bc9fa7c/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:02.602215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af8c07-121b-4dd0-b92f-9220a1aab810 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:02.602611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af8c07-121b-4dd0-b92f-9220a1aab810 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:02.603189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af8c07-121b-4dd0-b92f-9220a1aab810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:02.645171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-c4af8c07-121b-4dd0-b92f-9220a1aab810 None None] MFA Rules not processed for user `563dda8b2f42478eb5c903230bc9fa7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:02.646408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c4af8c07-121b-4dd0-b92f-9220a1aab810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:02.770370 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1044/2095] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:02.774988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50421d-b55c-4334-8fd7-5208c5d346a2 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:02.866224 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50421d-b55c-4334-8fd7-5208c5d346a2 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:02.872218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a038e7-4904-49b0-8677-7e13d0a6c155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:02.872833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a038e7-4904-49b0-8677-7e13d0a6c155 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:02.873124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a038e7-4904-49b0-8677-7e13d0a6c155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:02.873382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a038e7-4904-49b0-8677-7e13d0a6c155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:02.874022 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a038e7-4904-49b0-8677-7e13d0a6c155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:02.876002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a038e7-4904-49b0-8677-7e13d0a6c155 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:02.908164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1052/2096] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:02 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:02.909522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af8c07-121b-4dd0-b92f-9220a1aab810 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:02.920453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af8c07-121b-4dd0-b92f-9220a1aab810 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:02.978991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c3df7-f4eb-4b85-9ed3-5070fe8fbd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:02.979645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c3df7-f4eb-4b85-9ed3-5070fe8fbd9e admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:02.979926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c3df7-f4eb-4b85-9ed3-5070fe8fbd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:02.980194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c3df7-f4eb-4b85-9ed3-5070fe8fbd9e admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:02.980827 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c3df7-f4eb-4b85-9ed3-5070fe8fbd9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:02.983021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c3df7-f4eb-4b85-9ed3-5070fe8fbd9e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:03.007656 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1045/2097] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:02 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:03.012569 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a038e7-4904-49b0-8677-7e13d0a6c155 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:03.023208 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a038e7-4904-49b0-8677-7e13d0a6c155 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:03.027974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bf209-123b-45d3-9544-faeead9b1b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:03.028578 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bf209-123b-45d3-9544-faeead9b1b22 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:03.028851 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bf209-123b-45d3-9544-faeead9b1b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:03.029126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bf209-123b-45d3-9544-faeead9b1b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:03.029769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bf209-123b-45d3-9544-faeead9b1b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:03.031400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bf209-123b-45d3-9544-faeead9b1b22 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:03.043633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1053/2098] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:03.047866 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c3df7-f4eb-4b85-9ed3-5070fe8fbd9e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:03.059958 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c3df7-f4eb-4b85-9ed3-5070fe8fbd9e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:03.065441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f28465-4c4e-48d6-be62-e6c24dfa028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:03.066069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f28465-4c4e-48d6-be62-e6c24dfa028f admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:03.066347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f28465-4c4e-48d6-be62-e6c24dfa028f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:03.066624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f28465-4c4e-48d6-be62-e6c24dfa028f admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:03.067320 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f28465-4c4e-48d6-be62-e6c24dfa028f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:03.068977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f28465-4c4e-48d6-be62-e6c24dfa028f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:03.080989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1046/2099] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:03.092892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bf209-123b-45d3-9544-faeead9b1b22 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:03.103355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bf209-123b-45d3-9544-faeead9b1b22 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:03.109003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df181850-a329-4d7a-a2f7-ed069efbb040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:03.109673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df181850-a329-4d7a-a2f7-ed069efbb040 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:03.109955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df181850-a329-4d7a-a2f7-ed069efbb040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:03.110230 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df181850-a329-4d7a-a2f7-ed069efbb040 admin admin] PATH_INFO: `/v3/projects/355d713fdc5149289c1dd0d81d9f8001/users/5e86d20b02134d5daf2a7b1b9ed614e2/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:03.127446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df181850-a329-4d7a-a2f7-ed069efbb040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=355d713fdc5149289c1dd0d81d9f8001, user_id=5e86d20b02134d5daf2a7b1b9ed614e2, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:03.130369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df181850-a329-4d7a-a2f7-ed069efbb040 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:03.148892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1054/2100] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:03 2022] PUT /identity/v3/projects/355d713fdc5149289c1dd0d81d9f8001/users/5e86d20b02134d5daf2a7b1b9ed614e2/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:03.160062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a85c15-0558-41ba-ba7a-44ff168da249 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:03.160337 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a85c15-0558-41ba-ba7a-44ff168da249 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:03.160495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a85c15-0558-41ba-ba7a-44ff168da249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:03.192174 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:03.197661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-c7a85c15-0558-41ba-ba7a-44ff168da249 None None] MFA Rules not processed for user `5e86d20b02134d5daf2a7b1b9ed614e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:03.199005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c7a85c15-0558-41ba-ba7a-44ff168da249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:03.309304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1047/2101] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:03.314713 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df181850-a329-4d7a-a2f7-ed069efbb040 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:03.338907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:03.385233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df181850-a329-4d7a-a2f7-ed069efbb040 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:03.390449 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fca6fe-b53d-429a-9345-e002ce87c423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:03.391065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fca6fe-b53d-429a-9345-e002ce87c423 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:03.391373 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fca6fe-b53d-429a-9345-e002ce87c423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:03.391646 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fca6fe-b53d-429a-9345-e002ce87c423 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:03.392294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fca6fe-b53d-429a-9345-e002ce87c423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:03.394202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fca6fe-b53d-429a-9345-e002ce87c423 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:03.425079 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1055/2102] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:03 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:03.430082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a85c15-0558-41ba-ba7a-44ff168da249 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:03.441080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a85c15-0558-41ba-ba7a-44ff168da249 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:03.503548 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d94b9-d96d-4b5c-b6cb-18da8d7d7bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:03.504169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d94b9-d96d-4b5c-b6cb-18da8d7d7bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:03.504445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d94b9-d96d-4b5c-b6cb-18da8d7d7bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:03.504710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d94b9-d96d-4b5c-b6cb-18da8d7d7bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:03.505363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d94b9-d96d-4b5c-b6cb-18da8d7d7bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:03.507327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d94b9-d96d-4b5c-b6cb-18da8d7d7bd0 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:03.531218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1048/2103] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:03 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:03.535357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fca6fe-b53d-429a-9345-e002ce87c423 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:03.546376 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fca6fe-b53d-429a-9345-e002ce87c423 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:03.550992 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e6367-a40f-424b-a7b9-5f0f14404a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:03.551622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e6367-a40f-424b-a7b9-5f0f14404a7c admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:03.551897 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e6367-a40f-424b-a7b9-5f0f14404a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:03.552214 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e6367-a40f-424b-a7b9-5f0f14404a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:03.552811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e6367-a40f-424b-a7b9-5f0f14404a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:03.554439 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e6367-a40f-424b-a7b9-5f0f14404a7c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:03.565359 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1056/2104] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:18:03.569664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d94b9-d96d-4b5c-b6cb-18da8d7d7bd0 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:03.580027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d94b9-d96d-4b5c-b6cb-18da8d7d7bd0 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:03.584844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c123de2-69cb-4d4d-8fbf-7ee71e500d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:03.586057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c123de2-69cb-4d4d-8fbf-7ee71e500d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:03.586334 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c123de2-69cb-4d4d-8fbf-7ee71e500d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:03.586901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c123de2-69cb-4d4d-8fbf-7ee71e500d7a admin admin] PATH_INFO: `/v3/projects/8e5a57a0e89a462088a412c002519464/users/72482bb31ed24903b3bd6a48a677c156/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:03.605347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c123de2-69cb-4d4d-8fbf-7ee71e500d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e5a57a0e89a462088a412c002519464, user_id=72482bb31ed24903b3bd6a48a677c156, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:03.608851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c123de2-69cb-4d4d-8fbf-7ee71e500d7a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:03.622788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1049/2105] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:03 2022] PUT /identity/v3/projects/8e5a57a0e89a462088a412c002519464/users/72482bb31ed24903b3bd6a48a677c156/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:03.626916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e6367-a40f-424b-a7b9-5f0f14404a7c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:03.637153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e6367-a40f-424b-a7b9-5f0f14404a7c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:03.705671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c245e36-697e-44ab-8399-93defad40d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:03.706321 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c245e36-697e-44ab-8399-93defad40d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:03.706602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c245e36-697e-44ab-8399-93defad40d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:03.706879 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c245e36-697e-44ab-8399-93defad40d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:03.707613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c245e36-697e-44ab-8399-93defad40d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:03.709202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c245e36-697e-44ab-8399-93defad40d20 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:03.720194 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1057/2106] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:03.724481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c123de2-69cb-4d4d-8fbf-7ee71e500d7a admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:03.735815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c123de2-69cb-4d4d-8fbf-7ee71e500d7a admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:03.740499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e0b71-e1fc-4331-b02a-44d5cd07ce34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:03.741163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e0b71-e1fc-4331-b02a-44d5cd07ce34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:03.741472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e0b71-e1fc-4331-b02a-44d5cd07ce34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:03.741750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e0b71-e1fc-4331-b02a-44d5cd07ce34 admin admin] PATH_INFO: `/v3/projects/8e5a57a0e89a462088a412c002519464/users/72482bb31ed24903b3bd6a48a677c156/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:03.743690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e0b71-e1fc-4331-b02a-44d5cd07ce34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e5a57a0e89a462088a412c002519464, user_id=72482bb31ed24903b3bd6a48a677c156, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:03.746514 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e0b71-e1fc-4331-b02a-44d5cd07ce34 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:03.757155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1050/2107] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:03 2022] PUT /identity/v3/projects/8e5a57a0e89a462088a412c002519464/users/72482bb31ed24903b3bd6a48a677c156/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:03.767487 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59e114-cfa5-4e39-9fcc-877cee6edf62 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:03.767808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59e114-cfa5-4e39-9fcc-877cee6edf62 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:03.768080 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59e114-cfa5-4e39-9fcc-877cee6edf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:03.797616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-7d59e114-cfa5-4e39-9fcc-877cee6edf62 None None] MFA Rules not processed for user `72482bb31ed24903b3bd6a48a677c156`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:03.798933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7d59e114-cfa5-4e39-9fcc-877cee6edf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:03.889407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1058/2108] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:03.902187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e0b71-e1fc-4331-b02a-44d5cd07ce34 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:03.904765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2b0e0b71-e1fc-4331-b02a-44d5cd07ce34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:03.978921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e0b71-e1fc-4331-b02a-44d5cd07ce34 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:03.984950 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abf823-b62d-4f46-a6a7-ee7410b589e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:03.985923 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abf823-b62d-4f46-a6a7-ee7410b589e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:03.985923 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abf823-b62d-4f46-a6a7-ee7410b589e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:03.986248 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abf823-b62d-4f46-a6a7-ee7410b589e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:03.996689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:04.057676 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1abf823-b62d-4f46-a6a7-ee7410b589e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:04.059183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1abf823-b62d-4f46-a6a7-ee7410b589e8 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:04.068795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1051/2109] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:05.664988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59e114-cfa5-4e39-9fcc-877cee6edf62 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:05.666668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7d59e114-cfa5-4e39-9fcc-877cee6edf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:05.686864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59e114-cfa5-4e39-9fcc-877cee6edf62 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:05.691304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ad4ba-ddd8-443b-8e8c-f8daf0498533 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:05.691938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ad4ba-ddd8-443b-8e8c-f8daf0498533 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:05.692216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ad4ba-ddd8-443b-8e8c-f8daf0498533 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:05.692492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ad4ba-ddd8-443b-8e8c-f8daf0498533 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:05.714286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:05.778877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ad4ba-ddd8-443b-8e8c-f8daf0498533 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:05.780408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ad4ba-ddd8-443b-8e8c-f8daf0498533 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:05.790939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1059/2110] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:06.621584 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abf823-b62d-4f46-a6a7-ee7410b589e8 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:06.634500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abf823-b62d-4f46-a6a7-ee7410b589e8 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:06.641091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537c842-081c-4c09-9f09-25e133a2311d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:06.641802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537c842-081c-4c09-9f09-25e133a2311d service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:06.642147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537c842-081c-4c09-9f09-25e133a2311d service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:06.642513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537c842-081c-4c09-9f09-25e133a2311d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:06.654254 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537c842-081c-4c09-9f09-25e133a2311d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:06.656039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537c842-081c-4c09-9f09-25e133a2311d service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:06.675630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1052/2111] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:18:10.485221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ad4ba-ddd8-443b-8e8c-f8daf0498533 service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:10.488780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-6c3ad4ba-ddd8-443b-8e8c-f8daf0498533 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:10.610012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ad4ba-ddd8-443b-8e8c-f8daf0498533 service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:10.614867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:10.615568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:10.615922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:10.616202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] PATH_INFO: `/v3/users/4e2fa621eaf3452db62d884e65da7005` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:10.617473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e2fa621eaf3452db62d884e65da7005)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:10.619469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:10.676425 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e2fa621eaf3452db62d884e65da7005'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:10.693490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e2fa621eaf3452db62d884e65da7005'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:10.702969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e2fa621eaf3452db62d884e65da7005'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:10.704050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e2fa621eaf3452db62d884e65da7005'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:10.705024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e2fa621eaf3452db62d884e65da7005'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:10.725901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1060/2112] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:10 2022] DELETE /identity/v3/users/4e2fa621eaf3452db62d884e65da7005 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:10.731821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c71d3-33e2-498a-82e5-1416a4489f44 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:10.732619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c71d3-33e2-498a-82e5-1416a4489f44 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:10.732739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c71d3-33e2-498a-82e5-1416a4489f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:10.757061 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-e04c71d3-33e2-498a-82e5-1416a4489f44 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:10.758551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e04c71d3-33e2-498a-82e5-1416a4489f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:10.837388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:10.838904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:10.864698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1053/2113] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:10.875631 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c71d3-33e2-498a-82e5-1416a4489f44 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:10.877276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e04c71d3-33e2-498a-82e5-1416a4489f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:10.895352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:10.945654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0232038-ae85-4b23-b17d-55bc75fa81c8 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:10.950658 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:10.951339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:10.951737 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:10.952022 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] PATH_INFO: `/v3/users/563dda8b2f42478eb5c903230bc9fa7c` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:10.953222 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=563dda8b2f42478eb5c903230bc9fa7c)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:10.955202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:10.958820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c71d3-33e2-498a-82e5-1416a4489f44 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:10.963133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f180e-8dd1-45dc-8e47-dab83d712356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:10.963763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f180e-8dd1-45dc-8e47-dab83d712356 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:10.964048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f180e-8dd1-45dc-8e47-dab83d712356 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:10.964319 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f180e-8dd1-45dc-8e47-dab83d712356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:10.975386 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f180e-8dd1-45dc-8e47-dab83d712356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:10.976843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f180e-8dd1-45dc-8e47-dab83d712356 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:10.989658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1054/2114] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:11.018071 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563dda8b2f42478eb5c903230bc9fa7c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.038420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563dda8b2f42478eb5c903230bc9fa7c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.048532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563dda8b2f42478eb5c903230bc9fa7c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.049442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563dda8b2f42478eb5c903230bc9fa7c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.050148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563dda8b2f42478eb5c903230bc9fa7c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.074335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1061/2115] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:10 2022] DELETE /identity/v3/users/563dda8b2f42478eb5c903230bc9fa7c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:11.077192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf07afc-9b10-4c3f-9ffe-ec978dc2552a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:11.078019 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf07afc-9b10-4c3f-9ffe-ec978dc2552a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:11.078499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf07afc-9b10-4c3f-9ffe-ec978dc2552a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:11.118871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ccf07afc-9b10-4c3f-9ffe-ec978dc2552a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:11.121005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ccf07afc-9b10-4c3f-9ffe-ec978dc2552a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:11.167447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:11.199286 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1055/2116] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:11.209894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:11.211677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:11.276088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af59602-d57c-4144-a362-bd2193b5e7b2 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:11.281385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b8b29-39e5-4688-b02d-56171ce32b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:11.282100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8b29-39e5-4688-b02d-56171ce32b1d service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:11.282368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8b29-39e5-4688-b02d-56171ce32b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:11.282651 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8b29-39e5-4688-b02d-56171ce32b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:11.294763 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b8b29-39e5-4688-b02d-56171ce32b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:11.296722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b8b29-39e5-4688-b02d-56171ce32b1d service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:11.306905 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1062/2117] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:18:11.331988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf07afc-9b10-4c3f-9ffe-ec978dc2552a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:11.333619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ccf07afc-9b10-4c3f-9ffe-ec978dc2552a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:11.349533 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf07afc-9b10-4c3f-9ffe-ec978dc2552a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:11.360150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:11.360817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:11.361099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:11.361374 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] PATH_INFO: `/v3/projects/0d072d876a2549f8bc538c0e20d8a033` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:11.362580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d072d876a2549f8bc538c0e20d8a033)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:11.364908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:11.454703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d072d876a2549f8bc538c0e20d8a033', 'request_id': 'req-8faf68c7-1dd9-441e-8e31-926c8627aee4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.465714 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d072d876a2549f8bc538c0e20d8a033', 'request_id': 'req-8faf68c7-1dd9-441e-8e31-926c8627aee4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.472292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d072d876a2549f8bc538c0e20d8a033', 'request_id': 'req-8faf68c7-1dd9-441e-8e31-926c8627aee4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.486194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] The token cache is being invalidate because project 0d072d876a2549f8bc538c0e20d8a033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:11.486738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.489388 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.494006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1056/2118] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:11 2022] DELETE /identity/v3/projects/0d072d876a2549f8bc538c0e20d8a033 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:11.495735 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b8b29-39e5-4688-b02d-56171ce32b1d service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:11.508437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b8b29-39e5-4688-b02d-56171ce32b1d service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:11.601126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:11.611952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:11.613743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:11.619391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:11.619391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] PATH_INFO: `/v3/users/1eb4a1ce172e41c6967b003598247be1` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:11.619391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb4a1ce172e41c6967b003598247be1)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:11.622677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:11.719813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb4a1ce172e41c6967b003598247be1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.743052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb4a1ce172e41c6967b003598247be1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.755654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb4a1ce172e41c6967b003598247be1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.756724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb4a1ce172e41c6967b003598247be1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.759173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb4a1ce172e41c6967b003598247be1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:11.777123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1063/2119] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:11 2022] DELETE /identity/v3/users/1eb4a1ce172e41c6967b003598247be1 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:11.944459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:11.945918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:11.966555 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf68c7-1dd9-441e-8e31-926c8627aee4 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:12.099013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:12.100171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:12.101178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:12.101461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] PATH_INFO: `/v3/projects/3d9e7f5459a3498a917d83a169e4eec6` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:12.103148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d9e7f5459a3498a917d83a169e4eec6)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:12.104738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:12.166265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d9e7f5459a3498a917d83a169e4eec6', 'request_id': 'req-a9d80862-1077-4591-9f27-3187f9c29e9b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.182681 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d9e7f5459a3498a917d83a169e4eec6', 'request_id': 'req-a9d80862-1077-4591-9f27-3187f9c29e9b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.192873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d9e7f5459a3498a917d83a169e4eec6', 'request_id': 'req-a9d80862-1077-4591-9f27-3187f9c29e9b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.194643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:12.209063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] The token cache is being invalidate because project 3d9e7f5459a3498a917d83a169e4eec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:12.210854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.212324 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.215156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1057/2120] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:11 2022] DELETE /identity/v3/projects/3d9e7f5459a3498a917d83a169e4eec6 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:12.219418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:12.220896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:12.238485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3dabbf-9d8b-4409-a457-27c1b4493b4b admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:12.309003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:12.311989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:12.329436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357cb9d4-e4f3-45e6-9148-b79f939c5eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:12.330114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cb9d4-e4f3-45e6-9148-b79f939c5eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:12.330399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cb9d4-e4f3-45e6-9148-b79f939c5eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:12.330678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cb9d4-e4f3-45e6-9148-b79f939c5eb4 admin admin] PATH_INFO: `/v3/users/5e86d20b02134d5daf2a7b1b9ed614e2` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:12.332252 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357cb9d4-e4f3-45e6-9148-b79f939c5eb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e86d20b02134d5daf2a7b1b9ed614e2)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:12.334298 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357cb9d4-e4f3-45e6-9148-b79f939c5eb4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:12.419716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-357cb9d4-e4f3-45e6-9148-b79f939c5eb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e86d20b02134d5daf2a7b1b9ed614e2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.453242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-357cb9d4-e4f3-45e6-9148-b79f939c5eb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e86d20b02134d5daf2a7b1b9ed614e2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.462507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d80862-1077-4591-9f27-3187f9c29e9b admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:12.468051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16f98c-b707-4ef6-89d4-353ac60ae062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:12.468805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16f98c-b707-4ef6-89d4-353ac60ae062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:12.469090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16f98c-b707-4ef6-89d4-353ac60ae062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:12.469505 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16f98c-b707-4ef6-89d4-353ac60ae062 admin admin] PATH_INFO: `/v3/projects/53e5b8ce762340618890a12d9697853f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:12.469807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-357cb9d4-e4f3-45e6-9148-b79f939c5eb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e86d20b02134d5daf2a7b1b9ed614e2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.470626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-357cb9d4-e4f3-45e6-9148-b79f939c5eb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e86d20b02134d5daf2a7b1b9ed614e2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.471364 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-357cb9d4-e4f3-45e6-9148-b79f939c5eb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e86d20b02134d5daf2a7b1b9ed614e2'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.476086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16f98c-b707-4ef6-89d4-353ac60ae062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e5b8ce762340618890a12d9697853f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:12.478526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:12.481441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16f98c-b707-4ef6-89d4-353ac60ae062 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:12.508974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1064/2121] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:12 2022] DELETE /identity/v3/users/5e86d20b02134d5daf2a7b1b9ed614e2 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:12.595611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fe16f98c-b707-4ef6-89d4-353ac60ae062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e5b8ce762340618890a12d9697853f', 'request_id': 'req-fe16f98c-b707-4ef6-89d4-353ac60ae062'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.606354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fe16f98c-b707-4ef6-89d4-353ac60ae062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e5b8ce762340618890a12d9697853f', 'request_id': 'req-fe16f98c-b707-4ef6-89d4-353ac60ae062'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.613572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fe16f98c-b707-4ef6-89d4-353ac60ae062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e5b8ce762340618890a12d9697853f', 'request_id': 'req-fe16f98c-b707-4ef6-89d4-353ac60ae062'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.628507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fe16f98c-b707-4ef6-89d4-353ac60ae062 admin admin] The token cache is being invalidate because project 53e5b8ce762340618890a12d9697853f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:12.629156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fe16f98c-b707-4ef6-89d4-353ac60ae062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.629872 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fe16f98c-b707-4ef6-89d4-353ac60ae062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:12.632423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1058/2122] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:12 2022] DELETE /identity/v3/projects/53e5b8ce762340618890a12d9697853f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:12.638522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc59417-da44-4f4f-b81f-b975aec4fa37 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:12.638941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc59417-da44-4f4f-b81f-b975aec4fa37 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:12.639257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc59417-da44-4f4f-b81f-b975aec4fa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:12.666579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-ffc59417-da44-4f4f-b81f-b975aec4fa37 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:12.668070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ffc59417-da44-4f4f-b81f-b975aec4fa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:12.764292 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1065/2123] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:12.776933 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ebd64-e0f6-4295-bfd9-ecd1d47ff2c6 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:12.777283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ebd64-e0f6-4295-bfd9-ecd1d47ff2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:12.777526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ebd64-e0f6-4295-bfd9-ecd1d47ff2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:12.802014 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-2b9ebd64-e0f6-4295-bfd9-ecd1d47ff2c6 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:12.803339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2b9ebd64-e0f6-4295-bfd9-ecd1d47ff2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:12.818707 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1059/2124] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:12.818985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc59417-da44-4f4f-b81f-b975aec4fa37 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:12.829269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc59417-da44-4f4f-b81f-b975aec4fa37 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:12.836015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc11218-d5b6-42c9-a2a1-72ea98698cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:12.837019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc11218-d5b6-42c9-a2a1-72ea98698cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:12.837303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc11218-d5b6-42c9-a2a1-72ea98698cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:12.837642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc11218-d5b6-42c9-a2a1-72ea98698cf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:12.838506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc11218-d5b6-42c9-a2a1-72ea98698cf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:18:12.838623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc11218-d5b6-42c9-a2a1-72ea98698cf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:12.840427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc11218-d5b6-42c9-a2a1-72ea98698cf3 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:12.856569 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1066/2125] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:18:12.861387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ebd64-e0f6-4295-bfd9-ecd1d47ff2c6 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:12.875145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ebd64-e0f6-4295-bfd9-ecd1d47ff2c6 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:12.892173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80e4b6-cab4-48dc-a268-07d63d9b8320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:12.892813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80e4b6-cab4-48dc-a268-07d63d9b8320 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:12.893093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80e4b6-cab4-48dc-a268-07d63d9b8320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:12.893365 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80e4b6-cab4-48dc-a268-07d63d9b8320 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:12.894010 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80e4b6-cab4-48dc-a268-07d63d9b8320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:12.895994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80e4b6-cab4-48dc-a268-07d63d9b8320 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:12.934429 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1060/2126] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:12 2022] POST /identity/v3/projects => generated 361 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:12.938803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc11218-d5b6-42c9-a2a1-72ea98698cf3 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:12.955298 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc11218-d5b6-42c9-a2a1-72ea98698cf3 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:13.044594 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20581e5-e1bb-4152-b1d1-a982ffe479ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:13.045519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20581e5-e1bb-4152-b1d1-a982ffe479ec admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:13.045809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20581e5-e1bb-4152-b1d1-a982ffe479ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:13.046086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20581e5-e1bb-4152-b1d1-a982ffe479ec admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:13.046973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20581e5-e1bb-4152-b1d1-a982ffe479ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:13.049017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20581e5-e1bb-4152-b1d1-a982ffe479ec admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:13.089061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1067/2127] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:12 2022] POST /identity/v3/users => generated 281 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:13.090943 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80e4b6-cab4-48dc-a268-07d63d9b8320 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:13.115293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80e4b6-cab4-48dc-a268-07d63d9b8320 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:13.122982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd47679-3d35-4b7f-a41c-7718f482ef29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:13.124261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd47679-3d35-4b7f-a41c-7718f482ef29 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:13.124798 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd47679-3d35-4b7f-a41c-7718f482ef29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:13.125351 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd47679-3d35-4b7f-a41c-7718f482ef29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:13.126657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd47679-3d35-4b7f-a41c-7718f482ef29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:13.130621 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd47679-3d35-4b7f-a41c-7718f482ef29 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:13.152408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1061/2128] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:13.156798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20581e5-e1bb-4152-b1d1-a982ffe479ec admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:13.170061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20581e5-e1bb-4152-b1d1-a982ffe479ec admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:13.177541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70394225-d101-4e78-b216-8080c2224add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:13.178331 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70394225-d101-4e78-b216-8080c2224add admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:13.178615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70394225-d101-4e78-b216-8080c2224add admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:13.178892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70394225-d101-4e78-b216-8080c2224add admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:13.181399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70394225-d101-4e78-b216-8080c2224add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:13.186290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70394225-d101-4e78-b216-8080c2224add admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:13.201459 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1068/2129] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:13.205506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd47679-3d35-4b7f-a41c-7718f482ef29 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:13.217050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd47679-3d35-4b7f-a41c-7718f482ef29 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:13.223087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb71e9f-93d3-4a6e-8a91-7b791b850a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:13.223839 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb71e9f-93d3-4a6e-8a91-7b791b850a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:13.224164 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb71e9f-93d3-4a6e-8a91-7b791b850a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:13.224524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb71e9f-93d3-4a6e-8a91-7b791b850a22 admin admin] PATH_INFO: `/v3/projects/d23dee9fccd84779a8b685d30adbaa4a/users/450eda44350147efb617b173c8255b24/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:13.246316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb71e9f-93d3-4a6e-8a91-7b791b850a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23dee9fccd84779a8b685d30adbaa4a, user_id=450eda44350147efb617b173c8255b24, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:13.249501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb71e9f-93d3-4a6e-8a91-7b791b850a22 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:13.274956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1062/2130] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:13 2022] PUT /identity/v3/projects/d23dee9fccd84779a8b685d30adbaa4a/users/450eda44350147efb617b173c8255b24/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:13.287219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecfa83-76ca-4443-ac0e-b2a95268b376 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:13.288955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecfa83-76ca-4443-ac0e-b2a95268b376 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:13.289955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecfa83-76ca-4443-ac0e-b2a95268b376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:13.329664 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-bfecfa83-76ca-4443-ac0e-b2a95268b376 None None] MFA Rules not processed for user `450eda44350147efb617b173c8255b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:13.330925 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bfecfa83-76ca-4443-ac0e-b2a95268b376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:13.415326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:13.444663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1069/2131] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:13.485483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb71e9f-93d3-4a6e-8a91-7b791b850a22 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:13.488388 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8eb71e9f-93d3-4a6e-8a91-7b791b850a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:13.513560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:13.601073 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb71e9f-93d3-4a6e-8a91-7b791b850a22 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:13.605852 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fff7a7-3c94-42f2-9b1d-6926b4408b16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:13.606570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fff7a7-3c94-42f2-9b1d-6926b4408b16 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:13.607037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fff7a7-3c94-42f2-9b1d-6926b4408b16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:13.607704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fff7a7-3c94-42f2-9b1d-6926b4408b16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:13.642851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fff7a7-3c94-42f2-9b1d-6926b4408b16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:13.644346 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fff7a7-3c94-42f2-9b1d-6926b4408b16 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:13.675144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1063/2132] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:13.698154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecfa83-76ca-4443-ac0e-b2a95268b376 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:13.743931 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fff7a7-3c94-42f2-9b1d-6926b4408b16 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:13.745938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-04fff7a7-3c94-42f2-9b1d-6926b4408b16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:13.837000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecfa83-76ca-4443-ac0e-b2a95268b376 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:13.844631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46463a5-464d-43f8-bfcb-058b65075772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:13.845451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46463a5-464d-43f8-bfcb-058b65075772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:13.845563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46463a5-464d-43f8-bfcb-058b65075772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:13.845849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46463a5-464d-43f8-bfcb-058b65075772 admin admin] PATH_INFO: `/v3/users/450eda44350147efb617b173c8255b24` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:13.847044 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46463a5-464d-43f8-bfcb-058b65075772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=450eda44350147efb617b173c8255b24)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:13.849139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46463a5-464d-43f8-bfcb-058b65075772 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:13.887458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fff7a7-3c94-42f2-9b1d-6926b4408b16 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:13.893583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:13.894421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:13.894701 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:13.894970 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] PATH_INFO: `/v3/projects/355d713fdc5149289c1dd0d81d9f8001` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:13.896222 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=355d713fdc5149289c1dd0d81d9f8001)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:13.899071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:13.945068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a46463a5-464d-43f8-bfcb-058b65075772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '450eda44350147efb617b173c8255b24'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:13.979811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a46463a5-464d-43f8-bfcb-058b65075772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '450eda44350147efb617b173c8255b24'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:13.989040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a46463a5-464d-43f8-bfcb-058b65075772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '450eda44350147efb617b173c8255b24'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:13.990204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a46463a5-464d-43f8-bfcb-058b65075772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '450eda44350147efb617b173c8255b24'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:13.991442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a46463a5-464d-43f8-bfcb-058b65075772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '450eda44350147efb617b173c8255b24'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:14.004641 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '355d713fdc5149289c1dd0d81d9f8001', 'request_id': 'req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:14.018315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '355d713fdc5149289c1dd0d81d9f8001', 'request_id': 'req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:14.020771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1070/2133] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:13 2022] DELETE /identity/v3/users/450eda44350147efb617b173c8255b24 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:14.030158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '355d713fdc5149289c1dd0d81d9f8001', 'request_id': 'req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:14.038660 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f479b4-af73-4bb0-bc3f-f41fa1a73654 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:14.039186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f479b4-af73-4bb0-bc3f-f41fa1a73654 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:14.039569 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f479b4-af73-4bb0-bc3f-f41fa1a73654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:14.049398 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] The token cache is being invalidate because project 355d713fdc5149289c1dd0d81d9f8001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:14.050111 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:14.051769 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:14.054167 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1064/2134] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:13 2022] DELETE /identity/v3/projects/355d713fdc5149289c1dd0d81d9f8001 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:14.058072 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:14.060306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:14.066605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-c3f479b4-af73-4bb0-bc3f-f41fa1a73654 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:14.075647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c3f479b4-af73-4bb0-bc3f-f41fa1a73654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:14.133723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:14.153314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf470e-f8cc-48b8-afc8-15c6a47e5d6f admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:14.157060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1071/2135] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:14.160687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:14.161551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:14.161634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:14.161858 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] PATH_INFO: `/v3/users/72482bb31ed24903b3bd6a48a677c156` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:14.163172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=72482bb31ed24903b3bd6a48a677c156)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:14.165211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:14.171497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f479b4-af73-4bb0-bc3f-f41fa1a73654 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:14.173235 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c3f479b4-af73-4bb0-bc3f-f41fa1a73654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:14.188670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:14.250293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72482bb31ed24903b3bd6a48a677c156'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:14.260944 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72482bb31ed24903b3bd6a48a677c156'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:14.261835 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72482bb31ed24903b3bd6a48a677c156'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:14.263616 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72482bb31ed24903b3bd6a48a677c156'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:14.268464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f479b4-af73-4bb0-bc3f-f41fa1a73654 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:14.277901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7d62b-9abc-4296-9b16-9ef71cbec8b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:14.278557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7d62b-9abc-4296-9b16-9ef71cbec8b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:14.278842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7d62b-9abc-4296-9b16-9ef71cbec8b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:14.279172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7d62b-9abc-4296-9b16-9ef71cbec8b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:14.281560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-42a7d62b-9abc-4296-9b16-9ef71cbec8b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:14.297273 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72482bb31ed24903b3bd6a48a677c156'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:14.314861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1065/2136] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:14 2022] DELETE /identity/v3/users/72482bb31ed24903b3bd6a48a677c156 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:14.315742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7d62b-9abc-4296-9b16-9ef71cbec8b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:14.317261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7d62b-9abc-4296-9b16-9ef71cbec8b9 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:14.336592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1072/2137] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:18:15.015652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:15.017155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:15.098904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7d62b-9abc-4296-9b16-9ef71cbec8b9 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:15.103141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-42a7d62b-9abc-4296-9b16-9ef71cbec8b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:15.130194 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7d62b-9abc-4296-9b16-9ef71cbec8b9 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:15.135169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba69de8-bf6e-4e43-b483-9c6f5c46a07e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:15.140559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:15.141274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.141554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.141829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a admin admin] PATH_INFO: `/v3/projects/d23dee9fccd84779a8b685d30adbaa4a` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.142008 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4731b-cfcf-49fd-b39b-2837ef122238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:15.142640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4731b-cfcf-49fd-b39b-2837ef122238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.144904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d23dee9fccd84779a8b685d30adbaa4a)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:15.149505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:15.149630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4731b-cfcf-49fd-b39b-2837ef122238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.149922 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4731b-cfcf-49fd-b39b-2837ef122238 admin admin] PATH_INFO: `/v3/projects/8e5a57a0e89a462088a412c002519464` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.154639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4731b-cfcf-49fd-b39b-2837ef122238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e5a57a0e89a462088a412c002519464)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:15.159057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4731b-cfcf-49fd-b39b-2837ef122238 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:15.244746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-81d4731b-cfcf-49fd-b39b-2837ef122238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e5a57a0e89a462088a412c002519464', 'request_id': 'req-81d4731b-cfcf-49fd-b39b-2837ef122238'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:15.259987 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-81d4731b-cfcf-49fd-b39b-2837ef122238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e5a57a0e89a462088a412c002519464', 'request_id': 'req-81d4731b-cfcf-49fd-b39b-2837ef122238'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:15.269921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-81d4731b-cfcf-49fd-b39b-2837ef122238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e5a57a0e89a462088a412c002519464', 'request_id': 'req-81d4731b-cfcf-49fd-b39b-2837ef122238'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:15.273190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd23dee9fccd84779a8b685d30adbaa4a', 'request_id': 'req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:15.290071 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd23dee9fccd84779a8b685d30adbaa4a', 'request_id': 'req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:15.291060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-81d4731b-cfcf-49fd-b39b-2837ef122238 admin admin] The token cache is being invalidate because project 8e5a57a0e89a462088a412c002519464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:15.292105 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-81d4731b-cfcf-49fd-b39b-2837ef122238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:15.293544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-81d4731b-cfcf-49fd-b39b-2837ef122238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:15.298409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1066/2138] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:15 2022] DELETE /identity/v3/projects/8e5a57a0e89a462088a412c002519464 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:15.304143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2db8d8-2316-4989-a8eb-73261bac2a10 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.304645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2db8d8-2316-4989-a8eb-73261bac2a10 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.304930 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2db8d8-2316-4989-a8eb-73261bac2a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.306791 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd23dee9fccd84779a8b685d30adbaa4a', 'request_id': 'req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:15.332694 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:15.335450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a admin admin] The token cache is being invalidate because project d23dee9fccd84779a8b685d30adbaa4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:15.335636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:15.336256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-dfe79f65-1b0b-407d-86a6-1c9e5b4a270a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:15.341547 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1073/2139] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:15 2022] DELETE /identity/v3/projects/d23dee9fccd84779a8b685d30adbaa4a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:15.345156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c470c-c3e4-4dfc-af46-0f96c1fb0349 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.345473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c470c-c3e4-4dfc-af46-0f96c1fb0349 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.345796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c470c-c3e4-4dfc-af46-0f96c1fb0349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.370368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b86c470c-c3e4-4dfc-af46-0f96c1fb0349 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:15.371726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b86c470c-c3e4-4dfc-af46-0f96c1fb0349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:15.372303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-1a2db8d8-2316-4989-a8eb-73261bac2a10 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:15.373830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1a2db8d8-2316-4989-a8eb-73261bac2a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:15.479472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1074/2140] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:15.481013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f370324-606e-434c-a16c-10c57a160aae None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.481333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f370324-606e-434c-a16c-10c57a160aae None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.481585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f370324-606e-434c-a16c-10c57a160aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.498822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1067/2141] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:15.519015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d9340-8e07-40ff-8fd8-2b9531fe85cf None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.521566 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d9340-8e07-40ff-8fd8-2b9531fe85cf None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.522278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d9340-8e07-40ff-8fd8-2b9531fe85cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.532148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-9f370324-606e-434c-a16c-10c57a160aae None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:15.533410 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9f370324-606e-434c-a16c-10c57a160aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:15.556872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1075/2142] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:15.561447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f370324-606e-434c-a16c-10c57a160aae None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:15.571135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-6a9d9340-8e07-40ff-8fd8-2b9531fe85cf None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:15.573468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6a9d9340-8e07-40ff-8fd8-2b9531fe85cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:15.583709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f370324-606e-434c-a16c-10c57a160aae None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:15.592737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1068/2143] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:15.592934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e98f7d-0a2d-46a5-a4e0-44994bdb1e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:15.593588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e98f7d-0a2d-46a5-a4e0-44994bdb1e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.593723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d9340-8e07-40ff-8fd8-2b9531fe85cf None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:15.593887 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e98f7d-0a2d-46a5-a4e0-44994bdb1e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.594151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e98f7d-0a2d-46a5-a4e0-44994bdb1e97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.595642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e98f7d-0a2d-46a5-a4e0-44994bdb1e97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:18:15.595958 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e98f7d-0a2d-46a5-a4e0-44994bdb1e97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:15.598261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e98f7d-0a2d-46a5-a4e0-44994bdb1e97 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:15.604407 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d9340-8e07-40ff-8fd8-2b9531fe85cf None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:15.610374 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36a0a9-b1f4-4752-be23-ac24cfa5c336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:15.611024 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36a0a9-b1f4-4752-be23-ac24cfa5c336 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.611710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36a0a9-b1f4-4752-be23-ac24cfa5c336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.612015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36a0a9-b1f4-4752-be23-ac24cfa5c336 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.612957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36a0a9-b1f4-4752-be23-ac24cfa5c336 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:18:15.613262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36a0a9-b1f4-4752-be23-ac24cfa5c336 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:15.620020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36a0a9-b1f4-4752-be23-ac24cfa5c336 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:15.634011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1076/2144] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:18:15.634930 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1069/2145] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:18:15.640996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e98f7d-0a2d-46a5-a4e0-44994bdb1e97 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:15.642408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36a0a9-b1f4-4752-be23-ac24cfa5c336 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:15.662415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e98f7d-0a2d-46a5-a4e0-44994bdb1e97 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:15.668330 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f778d-66cf-4145-a6ec-9f18de07ff1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:15.669134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f778d-66cf-4145-a6ec-9f18de07ff1c admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.669516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f778d-66cf-4145-a6ec-9f18de07ff1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.669762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36a0a9-b1f4-4752-be23-ac24cfa5c336 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:15.670103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f778d-66cf-4145-a6ec-9f18de07ff1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.670933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f778d-66cf-4145-a6ec-9f18de07ff1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:15.676184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f778d-66cf-4145-a6ec-9f18de07ff1c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:15.682243 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2442589-b211-4d0f-8179-9c8aca54f874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:15.683119 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2442589-b211-4d0f-8179-9c8aca54f874 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.683423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2442589-b211-4d0f-8179-9c8aca54f874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.683783 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2442589-b211-4d0f-8179-9c8aca54f874 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.684632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2442589-b211-4d0f-8179-9c8aca54f874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:15.688304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2442589-b211-4d0f-8179-9c8aca54f874 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:15.725569 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:15.750541 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1070/2146] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:15 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:15.755314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2442589-b211-4d0f-8179-9c8aca54f874 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:15.764167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1077/2147] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:15 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:15.771311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f778d-66cf-4145-a6ec-9f18de07ff1c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:15.867536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2442589-b211-4d0f-8179-9c8aca54f874 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:15.870071 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f778d-66cf-4145-a6ec-9f18de07ff1c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:15.874080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4247d8c-178f-4ba9-9976-2cb96d574d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:15.874868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4247d8c-178f-4ba9-9976-2cb96d574d24 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.874981 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4247d8c-178f-4ba9-9976-2cb96d574d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.875296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4247d8c-178f-4ba9-9976-2cb96d574d24 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.875947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4247d8c-178f-4ba9-9976-2cb96d574d24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:15.877494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dddcd-f09e-47b1-a74e-83e3b50c2885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:15.878111 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4247d8c-178f-4ba9-9976-2cb96d574d24 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:15.878240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dddcd-f09e-47b1-a74e-83e3b50c2885 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.879035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dddcd-f09e-47b1-a74e-83e3b50c2885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.879373 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dddcd-f09e-47b1-a74e-83e3b50c2885 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.880030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dddcd-f09e-47b1-a74e-83e3b50c2885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:15.882594 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dddcd-f09e-47b1-a74e-83e3b50c2885 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:15.906557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1071/2148] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:15 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:15.908177 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4247d8c-178f-4ba9-9976-2cb96d574d24 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:15.914026 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1078/2149] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:15 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:15.920672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dddcd-f09e-47b1-a74e-83e3b50c2885 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:15.933214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4247d8c-178f-4ba9-9976-2cb96d574d24 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:15.937110 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dddcd-f09e-47b1-a74e-83e3b50c2885 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:15.940663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7f5c6-85af-436e-b454-a3b793935572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:15.941594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7f5c6-85af-436e-b454-a3b793935572 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.942050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7f5c6-85af-436e-b454-a3b793935572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.942486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7f5c6-85af-436e-b454-a3b793935572 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.943254 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd7f5c6-85af-436e-b454-a3b793935572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:15.943498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9c261-1d3d-40fb-8d39-a783136ff5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:15.944224 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9c261-1d3d-40fb-8d39-a783136ff5df admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.944505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9c261-1d3d-40fb-8d39-a783136ff5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.944781 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9c261-1d3d-40fb-8d39-a783136ff5df admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.945300 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd7f5c6-85af-436e-b454-a3b793935572 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:15.946359 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9c261-1d3d-40fb-8d39-a783136ff5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:15.948824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9c261-1d3d-40fb-8d39-a783136ff5df admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:15.956389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1072/2150] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:18:15.961090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7f5c6-85af-436e-b454-a3b793935572 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:15.964510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1079/2151] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:18:15.968913 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9c261-1d3d-40fb-8d39-a783136ff5df admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:15.974038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7f5c6-85af-436e-b454-a3b793935572 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:15.983334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9c261-1d3d-40fb-8d39-a783136ff5df admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:15.986803 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18687c-d491-484a-92cf-b3daea0b7387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:15.988097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18687c-d491-484a-92cf-b3daea0b7387 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.988787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18687c-d491-484a-92cf-b3daea0b7387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.989329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18687c-d491-484a-92cf-b3daea0b7387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.989465 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee55d9-38e5-4f1f-8394-948539988218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:15.990138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee55d9-38e5-4f1f-8394-948539988218 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:15.991322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18687c-d491-484a-92cf-b3daea0b7387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:15.991503 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee55d9-38e5-4f1f-8394-948539988218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:15.991774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee55d9-38e5-4f1f-8394-948539988218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:15.993117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee55d9-38e5-4f1f-8394-948539988218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:15.994782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18687c-d491-484a-92cf-b3daea0b7387 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:15.995097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee55d9-38e5-4f1f-8394-948539988218 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:16.019418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1080/2152] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:16.019617 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1073/2153] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:18:16.023174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee55d9-38e5-4f1f-8394-948539988218 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:16.023899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18687c-d491-484a-92cf-b3daea0b7387 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:16.043743 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee55d9-38e5-4f1f-8394-948539988218 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:16.044690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18687c-d491-484a-92cf-b3daea0b7387 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:16.049603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dee467-9173-4b11-9605-c77bcffeb594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:16.050372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dee467-9173-4b11-9605-c77bcffeb594 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:16.050700 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dee467-9173-4b11-9605-c77bcffeb594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:16.050860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d3d8a-8bda-48c3-b808-5ad31183b16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:16.051023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dee467-9173-4b11-9605-c77bcffeb594 admin admin] PATH_INFO: `/v3/projects/b00bc7cb62994c35a4bb13a9d3cba649/users/594513f53878404f96f065856094f219/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:16.051676 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3d8a-8bda-48c3-b808-5ad31183b16e admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:16.052022 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3d8a-8bda-48c3-b808-5ad31183b16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:16.052362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3d8a-8bda-48c3-b808-5ad31183b16e admin admin] PATH_INFO: `/v3/projects/ff7073fe1aef45dcb12c7ce5793b5783/users/1e221f292fb04353a08bf7bb5fc4ceb9/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:16.080622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dee467-9173-4b11-9605-c77bcffeb594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b00bc7cb62994c35a4bb13a9d3cba649, user_id=594513f53878404f96f065856094f219, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:16.082715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d3d8a-8bda-48c3-b808-5ad31183b16e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff7073fe1aef45dcb12c7ce5793b5783, user_id=1e221f292fb04353a08bf7bb5fc4ceb9, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:16.084140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dee467-9173-4b11-9605-c77bcffeb594 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:16.086652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d3d8a-8bda-48c3-b808-5ad31183b16e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:16.107664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1074/2154] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:16 2022] PUT /identity/v3/projects/ff7073fe1aef45dcb12c7ce5793b5783/users/1e221f292fb04353a08bf7bb5fc4ceb9/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:16.112146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1081/2155] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:16 2022] PUT /identity/v3/projects/b00bc7cb62994c35a4bb13a9d3cba649/users/594513f53878404f96f065856094f219/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:16.130404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08177875-f341-4cf4-aefa-cc81cd94f17d None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:16.130689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08177875-f341-4cf4-aefa-cc81cd94f17d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:16.130971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08177875-f341-4cf4-aefa-cc81cd94f17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:16.137629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9765f4d-92cf-4242-a763-86c67608ca25 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:16.137986 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9765f4d-92cf-4242-a763-86c67608ca25 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:16.138371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9765f4d-92cf-4242-a763-86c67608ca25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:16.163490 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:16.185643 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b9765f4d-92cf-4242-a763-86c67608ca25 None None] MFA Rules not processed for user `1e221f292fb04353a08bf7bb5fc4ceb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:16.187366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b9765f4d-92cf-4242-a763-86c67608ca25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:16.192711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-08177875-f341-4cf4-aefa-cc81cd94f17d None None] MFA Rules not processed for user `594513f53878404f96f065856094f219`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:16.193912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-08177875-f341-4cf4-aefa-cc81cd94f17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:16.290734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1075/2156] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:16.293635 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1082/2157] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:16.309581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08177875-f341-4cf4-aefa-cc81cd94f17d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:16.310880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9765f4d-92cf-4242-a763-86c67608ca25 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:16.311580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-08177875-f341-4cf4-aefa-cc81cd94f17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:16.312957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b9765f4d-92cf-4242-a763-86c67608ca25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:16.349405 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:16.397814 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9765f4d-92cf-4242-a763-86c67608ca25 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:16.402464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603d929-3cde-4e06-9c5d-764b38664062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:16.403607 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603d929-3cde-4e06-9c5d-764b38664062 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:16.403962 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603d929-3cde-4e06-9c5d-764b38664062 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:16.404238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603d929-3cde-4e06-9c5d-764b38664062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:16.416301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603d929-3cde-4e06-9c5d-764b38664062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:16.416997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08177875-f341-4cf4-aefa-cc81cd94f17d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:16.417896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603d929-3cde-4e06-9c5d-764b38664062 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:16.421377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece2786-3bf8-4b8b-8a41-8c86d5dba6d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:16.422291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece2786-3bf8-4b8b-8a41-8c86d5dba6d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:16.422613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece2786-3bf8-4b8b-8a41-8c86d5dba6d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:16.423343 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece2786-3bf8-4b8b-8a41-8c86d5dba6d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:16.428157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1083/2158] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:16.435302 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece2786-3bf8-4b8b-8a41-8c86d5dba6d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:16.437424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece2786-3bf8-4b8b-8a41-8c86d5dba6d0 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:16.448923 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1076/2159] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:18:17.876885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603d929-3cde-4e06-9c5d-764b38664062 service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:17.878614 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e603d929-3cde-4e06-9c5d-764b38664062 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:17.966346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603d929-3cde-4e06-9c5d-764b38664062 service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:17.971559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aba5fa-57c8-4da2-978e-5a92ca758dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:17.972219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aba5fa-57c8-4da2-978e-5a92ca758dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:17.972675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aba5fa-57c8-4da2-978e-5a92ca758dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:17.972978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aba5fa-57c8-4da2-978e-5a92ca758dad admin admin] PATH_INFO: `/v3/users/cf718a19fbc9458c996678ed7accc344` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:17.974399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aba5fa-57c8-4da2-978e-5a92ca758dad admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf718a19fbc9458c996678ed7accc344)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:17.977098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aba5fa-57c8-4da2-978e-5a92ca758dad admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:18.032034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-39aba5fa-57c8-4da2-978e-5a92ca758dad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf718a19fbc9458c996678ed7accc344'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:18.056237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-39aba5fa-57c8-4da2-978e-5a92ca758dad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf718a19fbc9458c996678ed7accc344'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:18.070200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-39aba5fa-57c8-4da2-978e-5a92ca758dad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf718a19fbc9458c996678ed7accc344'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:18.071448 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-39aba5fa-57c8-4da2-978e-5a92ca758dad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf718a19fbc9458c996678ed7accc344'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:18.072471 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-39aba5fa-57c8-4da2-978e-5a92ca758dad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf718a19fbc9458c996678ed7accc344'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:18.091348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1084/2160] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:17 2022] DELETE /identity/v3/users/cf718a19fbc9458c996678ed7accc344 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:18.332350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece2786-3bf8-4b8b-8a41-8c86d5dba6d0 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:18.344788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece2786-3bf8-4b8b-8a41-8c86d5dba6d0 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:18.425208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de30c8-3728-40b1-9811-813795061684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:18.425862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de30c8-3728-40b1-9811-813795061684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:18.426143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de30c8-3728-40b1-9811-813795061684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:18.426518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de30c8-3728-40b1-9811-813795061684 admin admin] PATH_INFO: `/v3/projects/d43169ae236d4341abb683010449c5e2` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:18.427635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de30c8-3728-40b1-9811-813795061684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d43169ae236d4341abb683010449c5e2)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:18.429746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de30c8-3728-40b1-9811-813795061684 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:18.493122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:18.504779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-09de30c8-3728-40b1-9811-813795061684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd43169ae236d4341abb683010449c5e2', 'request_id': 'req-09de30c8-3728-40b1-9811-813795061684'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:18.518204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-09de30c8-3728-40b1-9811-813795061684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd43169ae236d4341abb683010449c5e2', 'request_id': 'req-09de30c8-3728-40b1-9811-813795061684'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:18.530690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-09de30c8-3728-40b1-9811-813795061684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd43169ae236d4341abb683010449c5e2', 'request_id': 'req-09de30c8-3728-40b1-9811-813795061684'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:18.553962 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-09de30c8-3728-40b1-9811-813795061684 admin admin] The token cache is being invalidate because project d43169ae236d4341abb683010449c5e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:18.554512 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-09de30c8-3728-40b1-9811-813795061684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:18.555253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-09de30c8-3728-40b1-9811-813795061684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:18.556809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1077/2161] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:18 2022] DELETE /identity/v3/projects/d43169ae236d4341abb683010449c5e2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:18.561973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5030d0e-c93f-4025-bd9c-df7f439fb110 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:18.562475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5030d0e-c93f-4025-bd9c-df7f439fb110 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:18.562840 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5030d0e-c93f-4025-bd9c-df7f439fb110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:18.588325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b5030d0e-c93f-4025-bd9c-df7f439fb110 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:18.589571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b5030d0e-c93f-4025-bd9c-df7f439fb110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:18.650287 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:18.672979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1085/2162] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:18.682206 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e992b-0bb3-4cf9-a464-12fe57a54de0 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:18.682695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e992b-0bb3-4cf9-a464-12fe57a54de0 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:18.683233 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e992b-0bb3-4cf9-a464-12fe57a54de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:18.712010 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-3f6e992b-0bb3-4cf9-a464-12fe57a54de0 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:18.713446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3f6e992b-0bb3-4cf9-a464-12fe57a54de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:18.721563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1078/2163] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:18.725939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5030d0e-c93f-4025-bd9c-df7f439fb110 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:18.736515 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5030d0e-c93f-4025-bd9c-df7f439fb110 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:18.741955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9cd3c-7154-40f8-94c5-dc6aedd8a3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:18.742536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9cd3c-7154-40f8-94c5-dc6aedd8a3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:18.742820 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9cd3c-7154-40f8-94c5-dc6aedd8a3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:18.743132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9cd3c-7154-40f8-94c5-dc6aedd8a3a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:18.743792 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9cd3c-7154-40f8-94c5-dc6aedd8a3a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:18:18.744143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9cd3c-7154-40f8-94c5-dc6aedd8a3a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:18.745755 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9cd3c-7154-40f8-94c5-dc6aedd8a3a7 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:18.760894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1086/2164] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:18:18.765731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e992b-0bb3-4cf9-a464-12fe57a54de0 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:18.776328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e992b-0bb3-4cf9-a464-12fe57a54de0 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:18.782386 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff6457-44de-47f8-94dc-a35e351065b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:18.783222 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff6457-44de-47f8-94dc-a35e351065b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:18.783574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff6457-44de-47f8-94dc-a35e351065b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:18.784007 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff6457-44de-47f8-94dc-a35e351065b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:18.785104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff6457-44de-47f8-94dc-a35e351065b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:18.787977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff6457-44de-47f8-94dc-a35e351065b4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:18.822239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1079/2165] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:18 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:18.828311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9cd3c-7154-40f8-94c5-dc6aedd8a3a7 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:18.845058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9cd3c-7154-40f8-94c5-dc6aedd8a3a7 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:18.911712 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd8944-e352-4c51-b832-9940e5c30d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:18.912555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd8944-e352-4c51-b832-9940e5c30d05 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:18.912649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd8944-e352-4c51-b832-9940e5c30d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:18.912892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd8944-e352-4c51-b832-9940e5c30d05 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:18.913575 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd8944-e352-4c51-b832-9940e5c30d05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:18.915534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd8944-e352-4c51-b832-9940e5c30d05 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:18.938244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1087/2166] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:18 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:18.942699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff6457-44de-47f8-94dc-a35e351065b4 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:18.954223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff6457-44de-47f8-94dc-a35e351065b4 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:18.965796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2ceea-ebb8-4cc1-bc9a-1730e971563a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:18.966543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2ceea-ebb8-4cc1-bc9a-1730e971563a admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:18.966826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2ceea-ebb8-4cc1-bc9a-1730e971563a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:18.967224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2ceea-ebb8-4cc1-bc9a-1730e971563a admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:18.967899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2ceea-ebb8-4cc1-bc9a-1730e971563a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:18.969754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2ceea-ebb8-4cc1-bc9a-1730e971563a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:18.980906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1080/2167] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:18:18.985462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd8944-e352-4c51-b832-9940e5c30d05 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:18.996035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd8944-e352-4c51-b832-9940e5c30d05 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:19.001526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1f3f6-8811-4b3f-b357-8434a2a31a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:19.001526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1f3f6-8811-4b3f-b357-8434a2a31a7f admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:19.001760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1f3f6-8811-4b3f-b357-8434a2a31a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:19.002039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1f3f6-8811-4b3f-b357-8434a2a31a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:19.002694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1f3f6-8811-4b3f-b357-8434a2a31a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:19.004329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1f3f6-8811-4b3f-b357-8434a2a31a7f admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:19.019724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1088/2168] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:18:19.024450 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2ceea-ebb8-4cc1-bc9a-1730e971563a admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:19.036042 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2ceea-ebb8-4cc1-bc9a-1730e971563a admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:19.045147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f698f0-97e2-4ad9-83ca-65188aa5e919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:19.045147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f698f0-97e2-4ad9-83ca-65188aa5e919 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:19.045147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f698f0-97e2-4ad9-83ca-65188aa5e919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:19.045586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f698f0-97e2-4ad9-83ca-65188aa5e919 admin admin] PATH_INFO: `/v3/projects/34baa8df19ad40dfa447ef4f6ee35969/users/63b9687eeb3c45fe98808a0ebda40044/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:19.064353 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f698f0-97e2-4ad9-83ca-65188aa5e919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34baa8df19ad40dfa447ef4f6ee35969, user_id=63b9687eeb3c45fe98808a0ebda40044, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:19.067071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f698f0-97e2-4ad9-83ca-65188aa5e919 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:19.081687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1081/2169] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:19 2022] PUT /identity/v3/projects/34baa8df19ad40dfa447ef4f6ee35969/users/63b9687eeb3c45fe98808a0ebda40044/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:19.093342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16128746-0d0f-4101-807e-08111f2fcacf None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:19.093627 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16128746-0d0f-4101-807e-08111f2fcacf None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:19.093901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16128746-0d0f-4101-807e-08111f2fcacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:19.124400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-16128746-0d0f-4101-807e-08111f2fcacf None None] MFA Rules not processed for user `63b9687eeb3c45fe98808a0ebda40044`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:19.125634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-16128746-0d0f-4101-807e-08111f2fcacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:19.211779 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1089/2170] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:19.222695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f698f0-97e2-4ad9-83ca-65188aa5e919 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:19.224657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-61f698f0-97e2-4ad9-83ca-65188aa5e919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:19.319043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f698f0-97e2-4ad9-83ca-65188aa5e919 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:19.325276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c953e-af4e-4ef0-ac13-d0f909d29cf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:19.325926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c953e-af4e-4ef0-ac13-d0f909d29cf8 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:19.326209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c953e-af4e-4ef0-ac13-d0f909d29cf8 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:19.326486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c953e-af4e-4ef0-ac13-d0f909d29cf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:19.335857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c953e-af4e-4ef0-ac13-d0f909d29cf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:19.337481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c953e-af4e-4ef0-ac13-d0f909d29cf8 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:19.348806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1082/2171] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:29.205084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:29.342791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=71,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:41.829417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16128746-0d0f-4101-807e-08111f2fcacf None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:41.912375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16128746-0d0f-4101-807e-08111f2fcacf None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:41.918301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:41.919080 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:41.919407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:41.919937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] PATH_INFO: `/v3/users/63b9687eeb3c45fe98808a0ebda40044` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:41.921160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63b9687eeb3c45fe98808a0ebda40044)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:41.923302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:41.982054 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63b9687eeb3c45fe98808a0ebda40044'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:42.003592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63b9687eeb3c45fe98808a0ebda40044'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:42.011140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63b9687eeb3c45fe98808a0ebda40044'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:42.011907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63b9687eeb3c45fe98808a0ebda40044'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:42.012637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63b9687eeb3c45fe98808a0ebda40044'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:42.025200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1090/2172] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:41 2022] DELETE /identity/v3/users/63b9687eeb3c45fe98808a0ebda40044 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:42.030959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e72f8-d45a-47c5-a8ba-3554d04e291b None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:42.031558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e72f8-d45a-47c5-a8ba-3554d04e291b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:42.031558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e72f8-d45a-47c5-a8ba-3554d04e291b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:42.051185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-e31e72f8-d45a-47c5-a8ba-3554d04e291b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:42.052396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e31e72f8-d45a-47c5-a8ba-3554d04e291b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:42.116561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1083/2173] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:42.126659 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:42.128158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:42.193972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e56534-abdc-4ce8-a052-8f626c9be625 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:42.198481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032190fa-8552-4bc0-b69d-fc3ddcdc8ce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:42.199296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032190fa-8552-4bc0-b69d-fc3ddcdc8ce3 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:42.199589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032190fa-8552-4bc0-b69d-fc3ddcdc8ce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:42.199868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032190fa-8552-4bc0-b69d-fc3ddcdc8ce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:42.211349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032190fa-8552-4bc0-b69d-fc3ddcdc8ce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:42.212826 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032190fa-8552-4bc0-b69d-fc3ddcdc8ce3 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:42.224243 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1091/2174] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:42.516538 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e72f8-d45a-47c5-a8ba-3554d04e291b None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:42.518077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e31e72f8-d45a-47c5-a8ba-3554d04e291b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:42.533318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e72f8-d45a-47c5-a8ba-3554d04e291b None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:42.540911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8fb78-490a-49a5-bb7e-c44db3827f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:42.542206 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8fb78-490a-49a5-bb7e-c44db3827f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:42.542749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8fb78-490a-49a5-bb7e-c44db3827f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:42.543466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8fb78-490a-49a5-bb7e-c44db3827f70 admin admin] PATH_INFO: `/v3/projects/34baa8df19ad40dfa447ef4f6ee35969` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:42.545577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8fb78-490a-49a5-bb7e-c44db3827f70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34baa8df19ad40dfa447ef4f6ee35969)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:42.548243 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8fb78-490a-49a5-bb7e-c44db3827f70 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:42.611660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a3a8fb78-490a-49a5-bb7e-c44db3827f70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34baa8df19ad40dfa447ef4f6ee35969', 'request_id': 'req-a3a8fb78-490a-49a5-bb7e-c44db3827f70'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:42.632029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a3a8fb78-490a-49a5-bb7e-c44db3827f70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34baa8df19ad40dfa447ef4f6ee35969', 'request_id': 'req-a3a8fb78-490a-49a5-bb7e-c44db3827f70'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:42.638726 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a3a8fb78-490a-49a5-bb7e-c44db3827f70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34baa8df19ad40dfa447ef4f6ee35969', 'request_id': 'req-a3a8fb78-490a-49a5-bb7e-c44db3827f70'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:42.651436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a3a8fb78-490a-49a5-bb7e-c44db3827f70 admin admin] The token cache is being invalidate because project 34baa8df19ad40dfa447ef4f6ee35969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:42.651734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a3a8fb78-490a-49a5-bb7e-c44db3827f70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:42.652392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a3a8fb78-490a-49a5-bb7e-c44db3827f70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:42.653893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1084/2175] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:42 2022] DELETE /identity/v3/projects/34baa8df19ad40dfa447ef4f6ee35969 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:42.658670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ee6e7-bce4-45d3-8033-224d5610c7f0 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:42.659004 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ee6e7-bce4-45d3-8033-224d5610c7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:42.659306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ee6e7-bce4-45d3-8033-224d5610c7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:42.680379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-9a5ee6e7-bce4-45d3-8033-224d5610c7f0 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:42.681613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9a5ee6e7-bce4-45d3-8033-224d5610c7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:42.701217 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:42.749506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1092/2176] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:42.757979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a5454-0673-4a5e-9bb4-5e7b891d9474 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:42.758707 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a5454-0673-4a5e-9bb4-5e7b891d9474 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:42.758985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a5454-0673-4a5e-9bb4-5e7b891d9474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:42.780901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-fc6a5454-0673-4a5e-9bb4-5e7b891d9474 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:42.782112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-fc6a5454-0673-4a5e-9bb4-5e7b891d9474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:42.790821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1085/2177] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:42.795327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ee6e7-bce4-45d3-8033-224d5610c7f0 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:42.805663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ee6e7-bce4-45d3-8033-224d5610c7f0 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:42.810286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd5557-7913-4797-8b14-1fa8499670b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:42.810907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd5557-7913-4797-8b14-1fa8499670b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:42.811226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd5557-7913-4797-8b14-1fa8499670b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:42.811589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd5557-7913-4797-8b14-1fa8499670b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:42.812245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd5557-7913-4797-8b14-1fa8499670b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:18:42.812556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd5557-7913-4797-8b14-1fa8499670b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:42.814157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd5557-7913-4797-8b14-1fa8499670b0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:42.827483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1093/2178] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:18:42.831023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a5454-0673-4a5e-9bb4-5e7b891d9474 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:42.841265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a5454-0673-4a5e-9bb4-5e7b891d9474 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:42.848868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d73976-6460-4347-859a-660acec6a1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:42.850076 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d73976-6460-4347-859a-660acec6a1f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:42.850455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d73976-6460-4347-859a-660acec6a1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:42.850730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d73976-6460-4347-859a-660acec6a1f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:42.851472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d73976-6460-4347-859a-660acec6a1f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:42.853603 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d73976-6460-4347-859a-660acec6a1f0 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:42.886748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1086/2179] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:42 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:42.891322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd5557-7913-4797-8b14-1fa8499670b0 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:42.905040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd5557-7913-4797-8b14-1fa8499670b0 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:42.965224 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b1ea3-2bb1-45e9-aa53-78960894cf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:42.965836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b1ea3-2bb1-45e9-aa53-78960894cf14 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:42.966119 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b1ea3-2bb1-45e9-aa53-78960894cf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:42.966394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b1ea3-2bb1-45e9-aa53-78960894cf14 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:42.967040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b1ea3-2bb1-45e9-aa53-78960894cf14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:42.968995 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b1ea3-2bb1-45e9-aa53-78960894cf14 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:42.991500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1094/2180] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:42 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:42.995450 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d73976-6460-4347-859a-660acec6a1f0 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:43.006768 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d73976-6460-4347-859a-660acec6a1f0 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:43.013996 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58062f-9703-4783-9f17-83d6af650674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:43.014758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58062f-9703-4783-9f17-83d6af650674 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:43.015054 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58062f-9703-4783-9f17-83d6af650674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:43.015699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58062f-9703-4783-9f17-83d6af650674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:43.016339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58062f-9703-4783-9f17-83d6af650674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:43.018368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58062f-9703-4783-9f17-83d6af650674 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:43.031137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1087/2181] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:43.034011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b1ea3-2bb1-45e9-aa53-78960894cf14 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:43.043124 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58062f-9703-4783-9f17-83d6af650674 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:43.044402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-fd58062f-9703-4783-9f17-83d6af650674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:43.055233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b1ea3-2bb1-45e9-aa53-78960894cf14 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:43.055648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63c9fe-9f50-4b4e-a015-cc93a2b56b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:43.060972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63c9fe-9f50-4b4e-a015-cc93a2b56b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:43.060972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63c9fe-9f50-4b4e-a015-cc93a2b56b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:43.060972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63c9fe-9f50-4b4e-a015-cc93a2b56b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:43.060972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63c9fe-9f50-4b4e-a015-cc93a2b56b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:43.060972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63c9fe-9f50-4b4e-a015-cc93a2b56b55 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:43.061553 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58062f-9703-4783-9f17-83d6af650674 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:43.066234 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:43.067771 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:43.068054 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:43.068401 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] PATH_INFO: `/v3/users/1e221f292fb04353a08bf7bb5fc4ceb9` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:43.075144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1095/2182] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:43.075341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e221f292fb04353a08bf7bb5fc4ceb9)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:43.075425 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6c4a9-c58a-4499-a042-7a8db5aa8dc2 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:43.075425 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6c4a9-c58a-4499-a042-7a8db5aa8dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:43.075425 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6c4a9-c58a-4499-a042-7a8db5aa8dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:43.075634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:43.101788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-e4d6c4a9-c58a-4499-a042-7a8db5aa8dc2 None None] MFA Rules not processed for user `3f93418e3e264a1f8597f789ee888b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:43.103211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e4d6c4a9-c58a-4499-a042-7a8db5aa8dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:43.121125 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:43.128140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e221f292fb04353a08bf7bb5fc4ceb9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:43.137572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e221f292fb04353a08bf7bb5fc4ceb9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:43.138738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e221f292fb04353a08bf7bb5fc4ceb9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:43.139531 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e221f292fb04353a08bf7bb5fc4ceb9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:43.155354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e221f292fb04353a08bf7bb5fc4ceb9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:43.168578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1088/2183] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:43 2022] DELETE /identity/v3/users/1e221f292fb04353a08bf7bb5fc4ceb9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:43.171590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:43.172682 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:43.173288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1096/2184] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:43.174584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b34066-7594-4bfc-93bb-e128e1eec9ec None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:43.174927 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b34066-7594-4bfc-93bb-e128e1eec9ec None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:43.175192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b34066-7594-4bfc-93bb-e128e1eec9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:43.197430 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-00b34066-7594-4bfc-93bb-e128e1eec9ec None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:43.198682 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-00b34066-7594-4bfc-93bb-e128e1eec9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:43.235506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:43.260963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac25bb6-455b-45c7-a5fe-2615b32f125e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:43.265861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d532c0b-a643-4bc0-9983-c53146c723a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:43.266537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d532c0b-a643-4bc0-9983-c53146c723a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:43.266822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d532c0b-a643-4bc0-9983-c53146c723a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:43.267119 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d532c0b-a643-4bc0-9983-c53146c723a3 admin admin] PATH_INFO: `/v3/projects/db4fb633f434453e80616bdb14d021c4/users/11202ac5d2ad47029894e302abb3bd54/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:43.273785 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1097/2185] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:43.283680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b34066-7594-4bfc-93bb-e128e1eec9ec None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:43.285060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-00b34066-7594-4bfc-93bb-e128e1eec9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:43.290310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d532c0b-a643-4bc0-9983-c53146c723a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db4fb633f434453e80616bdb14d021c4, user_id=11202ac5d2ad47029894e302abb3bd54, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:43.293914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d532c0b-a643-4bc0-9983-c53146c723a3 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:43.311185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1089/2186] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:43 2022] PUT /identity/v3/projects/db4fb633f434453e80616bdb14d021c4/users/11202ac5d2ad47029894e302abb3bd54/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:43.320527 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f68ee-4c11-42c9-82db-2120e0e1f431 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:43.320814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f68ee-4c11-42c9-82db-2120e0e1f431 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:43.321086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f68ee-4c11-42c9-82db-2120e0e1f431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:43.351585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-967f68ee-4c11-42c9-82db-2120e0e1f431 None None] MFA Rules not processed for user `11202ac5d2ad47029894e302abb3bd54`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:43.352893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-967f68ee-4c11-42c9-82db-2120e0e1f431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:43.359593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b34066-7594-4bfc-93bb-e128e1eec9ec None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:43.436835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c4273-05b9-4f6c-bc88-70f300dc1779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:43.437565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c4273-05b9-4f6c-bc88-70f300dc1779 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:43.437933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c4273-05b9-4f6c-bc88-70f300dc1779 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:43.438268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c4273-05b9-4f6c-bc88-70f300dc1779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:43.438523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1090/2187] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:43.443128 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f68ee-4c11-42c9-82db-2120e0e1f431 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:43.488253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:43.532547 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c4273-05b9-4f6c-bc88-70f300dc1779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:43.533374 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f68ee-4c11-42c9-82db-2120e0e1f431 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:43.534474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c4273-05b9-4f6c-bc88-70f300dc1779 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:43.539200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc198c97-75c1-4142-b008-dce6dbaa4ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:43.540016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc198c97-75c1-4142-b008-dce6dbaa4ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:43.540298 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc198c97-75c1-4142-b008-dce6dbaa4ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:43.540570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc198c97-75c1-4142-b008-dce6dbaa4ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:43.541221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc198c97-75c1-4142-b008-dce6dbaa4ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:43.543173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc198c97-75c1-4142-b008-dce6dbaa4ea7 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:43.546562 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1098/2188] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:18:43.589366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1091/2189] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:43 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:43.593232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c4273-05b9-4f6c-bc88-70f300dc1779 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:43.608569 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c4273-05b9-4f6c-bc88-70f300dc1779 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:43.682198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:43.719869 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efe565-5d60-47ed-925c-92e56ae92579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:43.720694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efe565-5d60-47ed-925c-92e56ae92579 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:43.721046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efe565-5d60-47ed-925c-92e56ae92579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:43.721392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efe565-5d60-47ed-925c-92e56ae92579 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:43.722313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efe565-5d60-47ed-925c-92e56ae92579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:43.726537 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efe565-5d60-47ed-925c-92e56ae92579 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:43.763018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1099/2190] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:43 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:43.765560 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc198c97-75c1-4142-b008-dce6dbaa4ea7 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:43.781802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc198c97-75c1-4142-b008-dce6dbaa4ea7 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:43.798293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760cd30-2155-41ac-8989-a90fa3e54d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:43.800173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760cd30-2155-41ac-8989-a90fa3e54d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:43.801150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760cd30-2155-41ac-8989-a90fa3e54d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:43.802455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760cd30-2155-41ac-8989-a90fa3e54d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:43.803979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760cd30-2155-41ac-8989-a90fa3e54d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:43.812801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760cd30-2155-41ac-8989-a90fa3e54d8f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:43.837646 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1092/2191] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:43.843085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efe565-5d60-47ed-925c-92e56ae92579 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:43.867841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efe565-5d60-47ed-925c-92e56ae92579 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:43.877623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324476f-219e-492d-a67f-e585dd86ab0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:43.878790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324476f-219e-492d-a67f-e585dd86ab0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:43.879353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324476f-219e-492d-a67f-e585dd86ab0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:43.879860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324476f-219e-492d-a67f-e585dd86ab0f admin admin] PATH_INFO: `/v3/projects/22dc65ed9d9d4f46a07f7487eb83d799/users/333f7dac49654c86b910597c157a8572/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:43.910174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324476f-219e-492d-a67f-e585dd86ab0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=22dc65ed9d9d4f46a07f7487eb83d799, user_id=333f7dac49654c86b910597c157a8572, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:43.915454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324476f-219e-492d-a67f-e585dd86ab0f admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:43.933347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1100/2192] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:43 2022] PUT /identity/v3/projects/22dc65ed9d9d4f46a07f7487eb83d799/users/333f7dac49654c86b910597c157a8572/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:43.939711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760cd30-2155-41ac-8989-a90fa3e54d8f admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:43.950301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760cd30-2155-41ac-8989-a90fa3e54d8f admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:44.054057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f80b47-b646-421f-9ffe-af1a8e0bfb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:44.057199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f80b47-b646-421f-9ffe-af1a8e0bfb35 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.058060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f80b47-b646-421f-9ffe-af1a8e0bfb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.058060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f80b47-b646-421f-9ffe-af1a8e0bfb35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.058490 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f80b47-b646-421f-9ffe-af1a8e0bfb35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:44.060208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f80b47-b646-421f-9ffe-af1a8e0bfb35 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:44.087821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324476f-219e-492d-a67f-e585dd86ab0f admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:44.101205 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9324476f-219e-492d-a67f-e585dd86ab0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:44.101564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1093/2193] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:18:44.101564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f80b47-b646-421f-9ffe-af1a8e0bfb35 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:44.111097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f80b47-b646-421f-9ffe-af1a8e0bfb35 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:44.117902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32f3f0-c490-42a7-813d-1812b85a2780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:44.118722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32f3f0-c490-42a7-813d-1812b85a2780 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.119097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32f3f0-c490-42a7-813d-1812b85a2780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.119536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32f3f0-c490-42a7-813d-1812b85a2780 admin admin] PATH_INFO: `/v3/projects/22dc65ed9d9d4f46a07f7487eb83d799/users/333f7dac49654c86b910597c157a8572/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.121524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32f3f0-c490-42a7-813d-1812b85a2780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22dc65ed9d9d4f46a07f7487eb83d799, user_id=333f7dac49654c86b910597c157a8572, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:44.124596 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32f3f0-c490-42a7-813d-1812b85a2780 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:44.137673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324476f-219e-492d-a67f-e585dd86ab0f admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:44.143589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1094/2194] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:44 2022] PUT /identity/v3/projects/22dc65ed9d9d4f46a07f7487eb83d799/users/333f7dac49654c86b910597c157a8572/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:44.156229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb2723-f73d-47da-bf2b-19dc52b4a6dc None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.157204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb2723-f73d-47da-bf2b-19dc52b4a6dc None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.157470 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb2723-f73d-47da-bf2b-19dc52b4a6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.191485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-77fb2723-f73d-47da-bf2b-19dc52b4a6dc None None] MFA Rules not processed for user `333f7dac49654c86b910597c157a8572`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:44.192818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-77fb2723-f73d-47da-bf2b-19dc52b4a6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:44.221076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d1289-312c-4d71-80e7-2f2c7b685bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:44.222029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d1289-312c-4d71-80e7-2f2c7b685bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.223631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d1289-312c-4d71-80e7-2f2c7b685bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.224009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d1289-312c-4d71-80e7-2f2c7b685bcc admin admin] PATH_INFO: `/v3/projects/ff7073fe1aef45dcb12c7ce5793b5783` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.226468 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d1289-312c-4d71-80e7-2f2c7b685bcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff7073fe1aef45dcb12c7ce5793b5783)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:44.230569 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d1289-312c-4d71-80e7-2f2c7b685bcc admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:44.311430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1095/2195] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:44.316497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f30d1289-312c-4d71-80e7-2f2c7b685bcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff7073fe1aef45dcb12c7ce5793b5783', 'request_id': 'req-f30d1289-312c-4d71-80e7-2f2c7b685bcc'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.321610 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb2723-f73d-47da-bf2b-19dc52b4a6dc None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:44.323524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-77fb2723-f73d-47da-bf2b-19dc52b4a6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:44.325519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f30d1289-312c-4d71-80e7-2f2c7b685bcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff7073fe1aef45dcb12c7ce5793b5783', 'request_id': 'req-f30d1289-312c-4d71-80e7-2f2c7b685bcc'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.333914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f30d1289-312c-4d71-80e7-2f2c7b685bcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff7073fe1aef45dcb12c7ce5793b5783', 'request_id': 'req-f30d1289-312c-4d71-80e7-2f2c7b685bcc'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.344255 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:44.359971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f30d1289-312c-4d71-80e7-2f2c7b685bcc admin admin] The token cache is being invalidate because project ff7073fe1aef45dcb12c7ce5793b5783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:44.360447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f30d1289-312c-4d71-80e7-2f2c7b685bcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.361873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f30d1289-312c-4d71-80e7-2f2c7b685bcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:44.370170 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1101/2196] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:44 2022] DELETE /identity/v3/projects/ff7073fe1aef45dcb12c7ce5793b5783 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:44.374844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250904f4-44e9-40d1-b072-edc007f6a96d None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.375265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250904f4-44e9-40d1-b072-edc007f6a96d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.375772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250904f4-44e9-40d1-b072-edc007f6a96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.400221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-250904f4-44e9-40d1-b072-edc007f6a96d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:44.401476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-250904f4-44e9-40d1-b072-edc007f6a96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:44.417582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb2723-f73d-47da-bf2b-19dc52b4a6dc None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:44.441018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:44.478136 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1102/2197] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:44.478672 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9123a-97e8-4997-b24d-991d8156e330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:44.479501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9123a-97e8-4997-b24d-991d8156e330 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.479744 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9123a-97e8-4997-b24d-991d8156e330 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.480023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9123a-97e8-4997-b24d-991d8156e330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.482394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6ad9123a-97e8-4997-b24d-991d8156e330 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:44.487539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b51bb-f2a2-4d23-8aec-2632bcf69b5f None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.487824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b51bb-f2a2-4d23-8aec-2632bcf69b5f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.488095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b51bb-f2a2-4d23-8aec-2632bcf69b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.510794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-2f9b51bb-f2a2-4d23-8aec-2632bcf69b5f None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:44.512065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2f9b51bb-f2a2-4d23-8aec-2632bcf69b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:44.520695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1103/2198] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:44.525126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b51bb-f2a2-4d23-8aec-2632bcf69b5f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:44.535058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b51bb-f2a2-4d23-8aec-2632bcf69b5f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:44.540873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e6d22-9204-48e7-83e7-27a3aac3444a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:44.541490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e6d22-9204-48e7-83e7-27a3aac3444a admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.541768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e6d22-9204-48e7-83e7-27a3aac3444a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.542044 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e6d22-9204-48e7-83e7-27a3aac3444a admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.542717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e6d22-9204-48e7-83e7-27a3aac3444a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:18:44.543019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e6d22-9204-48e7-83e7-27a3aac3444a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:44.544768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e6d22-9204-48e7-83e7-27a3aac3444a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:44.557690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1104/2199] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:18:44.561424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9123a-97e8-4997-b24d-991d8156e330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:44.562168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e6d22-9204-48e7-83e7-27a3aac3444a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:44.563027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9123a-97e8-4997-b24d-991d8156e330 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:44.574305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1096/2200] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:18:44.578876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e6d22-9204-48e7-83e7-27a3aac3444a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:44.584118 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cad8e5-7756-421e-bb91-33cf89bd33c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:44.584868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cad8e5-7756-421e-bb91-33cf89bd33c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.585304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cad8e5-7756-421e-bb91-33cf89bd33c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.585659 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cad8e5-7756-421e-bb91-33cf89bd33c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.586962 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cad8e5-7756-421e-bb91-33cf89bd33c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:44.589196 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cad8e5-7756-421e-bb91-33cf89bd33c7 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:44.626373 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1105/2201] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:44 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:44.632863 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9123a-97e8-4997-b24d-991d8156e330 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:44.649183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9123a-97e8-4997-b24d-991d8156e330 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:44.717414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17425f07-cf74-4fea-b5d3-cfcc8bc89ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:44.718185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17425f07-cf74-4fea-b5d3-cfcc8bc89ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.718496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17425f07-cf74-4fea-b5d3-cfcc8bc89ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.718848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17425f07-cf74-4fea-b5d3-cfcc8bc89ad1 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.719528 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17425f07-cf74-4fea-b5d3-cfcc8bc89ad1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:44.721621 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17425f07-cf74-4fea-b5d3-cfcc8bc89ad1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:44.759725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1097/2202] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:44 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:44.763937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cad8e5-7756-421e-bb91-33cf89bd33c7 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:44.778295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cad8e5-7756-421e-bb91-33cf89bd33c7 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:44.784244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b25062-b2c5-4f1f-98fb-07d52825a5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:44.785166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b25062-b2c5-4f1f-98fb-07d52825a5ba admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.787175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b25062-b2c5-4f1f-98fb-07d52825a5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.787573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b25062-b2c5-4f1f-98fb-07d52825a5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.788303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b25062-b2c5-4f1f-98fb-07d52825a5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:44.790171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b25062-b2c5-4f1f-98fb-07d52825a5ba admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:44.801097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1106/2203] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:18:44.805353 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17425f07-cf74-4fea-b5d3-cfcc8bc89ad1 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:44.819369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17425f07-cf74-4fea-b5d3-cfcc8bc89ad1 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:44.824851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4241ecc-01bb-4f11-824b-0bf07fc61a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:44.825471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4241ecc-01bb-4f11-824b-0bf07fc61a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.825750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4241ecc-01bb-4f11-824b-0bf07fc61a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.826021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4241ecc-01bb-4f11-824b-0bf07fc61a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.826875 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4241ecc-01bb-4f11-824b-0bf07fc61a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:44.828581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4241ecc-01bb-4f11-824b-0bf07fc61a5f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:44.831657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:44.852844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1098/2204] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:44.857217 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b25062-b2c5-4f1f-98fb-07d52825a5ba admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:44.868821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b25062-b2c5-4f1f-98fb-07d52825a5ba admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:44.880375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55422c-1b50-4ea9-963d-1025feef716d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:44.881654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55422c-1b50-4ea9-963d-1025feef716d admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.882241 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55422c-1b50-4ea9-963d-1025feef716d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.882536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55422c-1b50-4ea9-963d-1025feef716d admin admin] PATH_INFO: `/v3/projects/d916ed5ea0874a0f85089a99d4a66c12/users/d6083ed15f884ffea8550f28addcbfcd/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:44.937531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55422c-1b50-4ea9-963d-1025feef716d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d916ed5ea0874a0f85089a99d4a66c12, user_id=d6083ed15f884ffea8550f28addcbfcd, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:44.942068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55422c-1b50-4ea9-963d-1025feef716d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:44.984649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1107/2205] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:44 2022] PUT /identity/v3/projects/d916ed5ea0874a0f85089a99d4a66c12/users/d6083ed15f884ffea8550f28addcbfcd/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:44.989995 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272e940-5939-4115-b2cb-e94815a30df1 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:44.990371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272e940-5939-4115-b2cb-e94815a30df1 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:44.990709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272e940-5939-4115-b2cb-e94815a30df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:45.038431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-d272e940-5939-4115-b2cb-e94815a30df1 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:45.041486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d272e940-5939-4115-b2cb-e94815a30df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:45.154994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1099/2206] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:45.166469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55422c-1b50-4ea9-963d-1025feef716d admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:45.168821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-0c55422c-1b50-4ea9-963d-1025feef716d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:45.262975 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55422c-1b50-4ea9-963d-1025feef716d admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:45.266433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70a34c-c3a6-47a5-b979-799632f55c0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:45.267096 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70a34c-c3a6-47a5-b979-799632f55c0a service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:45.267414 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70a34c-c3a6-47a5-b979-799632f55c0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:45.267935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70a34c-c3a6-47a5-b979-799632f55c0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:45.281127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70a34c-c3a6-47a5-b979-799632f55c0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:45.282227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70a34c-c3a6-47a5-b979-799632f55c0a service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:45.303328 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1108/2207] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:46.031989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7734cf-3750-425c-a81a-1e6122b2f46a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:46.032308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7734cf-3750-425c-a81a-1e6122b2f46a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:46.032648 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7734cf-3750-425c-a81a-1e6122b2f46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:46.071070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-bc7734cf-3750-425c-a81a-1e6122b2f46a None None] MFA Rules not processed for user `d6083ed15f884ffea8550f28addcbfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:46.072295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-bc7734cf-3750-425c-a81a-1e6122b2f46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:46.120016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70a34c-c3a6-47a5-b979-799632f55c0a service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:46.122220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ae70a34c-c3a6-47a5-b979-799632f55c0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:46.169176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70a34c-c3a6-47a5-b979-799632f55c0a service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:46.171127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1100/2208] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:46.180670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:46.181762 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:46.182979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:46.183316 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] PATH_INFO: `/v3/users/594513f53878404f96f065856094f219` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:46.183558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7734cf-3750-425c-a81a-1e6122b2f46a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:46.185726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=594513f53878404f96f065856094f219)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:46.188388 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-bc7734cf-3750-425c-a81a-1e6122b2f46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:46.188758 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:46.321982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7734cf-3750-425c-a81a-1e6122b2f46a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:46.325432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '594513f53878404f96f065856094f219'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:46.326291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f377fb-a76e-4da7-8d79-4e70c141b3e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:46.327013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f377fb-a76e-4da7-8d79-4e70c141b3e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:46.327407 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f377fb-a76e-4da7-8d79-4e70c141b3e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:46.327750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f377fb-a76e-4da7-8d79-4e70c141b3e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:46.339826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f377fb-a76e-4da7-8d79-4e70c141b3e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:46.341580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f377fb-a76e-4da7-8d79-4e70c141b3e6 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:46.352847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:46.371061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '594513f53878404f96f065856094f219'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:46.382329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '594513f53878404f96f065856094f219'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:46.383090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '594513f53878404f96f065856094f219'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:46.383780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '594513f53878404f96f065856094f219'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:46.413896 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1101/2209] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:18:46.423969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1109/2210] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:46 2022] DELETE /identity/v3/users/594513f53878404f96f065856094f219 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:46.429668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a5e42-fa7e-4cb3-b327-1e08b235f1fd None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:46.429668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a5e42-fa7e-4cb3-b327-1e08b235f1fd None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:46.429668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a5e42-fa7e-4cb3-b327-1e08b235f1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:46.452639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ed3a5e42-fa7e-4cb3-b327-1e08b235f1fd None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:46.454093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ed3a5e42-fa7e-4cb3-b327-1e08b235f1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:46.484320 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:46.529491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1102/2211] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:46.563998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:46.566697 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:46.637634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a5e42-fa7e-4cb3-b327-1e08b235f1fd None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:46.639963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ed3a5e42-fa7e-4cb3-b327-1e08b235f1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:46.667541 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a5e42-fa7e-4cb3-b327-1e08b235f1fd None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:46.677341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:46.678302 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:46.678775 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:46.679237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] PATH_INFO: `/v3/users/11202ac5d2ad47029894e302abb3bd54` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:46.690930 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11202ac5d2ad47029894e302abb3bd54)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:46.692411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a3770-ae01-45bf-8ada-c8db9163b9d9 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:46.693830 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:46.700371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2892b84-2d68-4202-8f80-4dc910701a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:46.701142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2892b84-2d68-4202-8f80-4dc910701a44 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:46.701711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2892b84-2d68-4202-8f80-4dc910701a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:46.701989 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2892b84-2d68-4202-8f80-4dc910701a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:46.730511 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2892b84-2d68-4202-8f80-4dc910701a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:46.732540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2892b84-2d68-4202-8f80-4dc910701a44 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:46.765399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1110/2212] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:46.792924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11202ac5d2ad47029894e302abb3bd54'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:46.804507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11202ac5d2ad47029894e302abb3bd54'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:46.805725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11202ac5d2ad47029894e302abb3bd54'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:46.807133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11202ac5d2ad47029894e302abb3bd54'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:46.835367 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11202ac5d2ad47029894e302abb3bd54'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:46.857137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1103/2213] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:46 2022] DELETE /identity/v3/users/11202ac5d2ad47029894e302abb3bd54 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:46.864063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00402ed-2a9c-403b-83ae-c579bb1e210f None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:46.864468 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00402ed-2a9c-403b-83ae-c579bb1e210f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:46.864858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00402ed-2a9c-403b-83ae-c579bb1e210f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:46.910411 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-c00402ed-2a9c-403b-83ae-c579bb1e210f None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:46.912189 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c00402ed-2a9c-403b-83ae-c579bb1e210f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:47.056384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1111/2214] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:47.080481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:47.082478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:47.172178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7886bcc-8cf9-4dbf-bc33-4ec780c39558 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:47.177520 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305fe45-9573-4a73-b6d5-3b9e48e37258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:47.178158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305fe45-9573-4a73-b6d5-3b9e48e37258 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:47.178437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305fe45-9573-4a73-b6d5-3b9e48e37258 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:47.178709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305fe45-9573-4a73-b6d5-3b9e48e37258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:47.218969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305fe45-9573-4a73-b6d5-3b9e48e37258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:47.220547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305fe45-9573-4a73-b6d5-3b9e48e37258 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:47.245320 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1104/2215] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:47.304666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00402ed-2a9c-403b-83ae-c579bb1e210f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:47.307179 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c00402ed-2a9c-403b-83ae-c579bb1e210f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:47.350810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00402ed-2a9c-403b-83ae-c579bb1e210f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:47.373166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:47.373933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:47.374228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:47.374539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] PATH_INFO: `/v3/projects/b00bc7cb62994c35a4bb13a9d3cba649` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:47.375918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b00bc7cb62994c35a4bb13a9d3cba649)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:47.378269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:47.484424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b00bc7cb62994c35a4bb13a9d3cba649', 'request_id': 'req-eeb82306-bf4d-4622-83fa-d718b4be645c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:47.496820 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b00bc7cb62994c35a4bb13a9d3cba649', 'request_id': 'req-eeb82306-bf4d-4622-83fa-d718b4be645c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:47.523140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b00bc7cb62994c35a4bb13a9d3cba649', 'request_id': 'req-eeb82306-bf4d-4622-83fa-d718b4be645c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:47.549548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:47.556999 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] The token cache is being invalidate because project b00bc7cb62994c35a4bb13a9d3cba649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:47.557626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:47.559229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:47.563243 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1112/2216] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:47 2022] DELETE /identity/v3/projects/b00bc7cb62994c35a4bb13a9d3cba649 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:47.699226 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305fe45-9573-4a73-b6d5-3b9e48e37258 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:47.701233 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1305fe45-9573-4a73-b6d5-3b9e48e37258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:47.748871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305fe45-9573-4a73-b6d5-3b9e48e37258 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:47.805526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:47.826832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa05097-3971-498f-9d68-a9e540b50e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:47.827534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa05097-3971-498f-9d68-a9e540b50e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:47.827813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa05097-3971-498f-9d68-a9e540b50e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:47.828091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa05097-3971-498f-9d68-a9e540b50e92 admin admin] PATH_INFO: `/v3/projects/db4fb633f434453e80616bdb14d021c4` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:47.829230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa05097-3971-498f-9d68-a9e540b50e92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db4fb633f434453e80616bdb14d021c4)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:47.831194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa05097-3971-498f-9d68-a9e540b50e92 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:47.885727 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ffa05097-3971-498f-9d68-a9e540b50e92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db4fb633f434453e80616bdb14d021c4', 'request_id': 'req-ffa05097-3971-498f-9d68-a9e540b50e92'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:47.895379 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ffa05097-3971-498f-9d68-a9e540b50e92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db4fb633f434453e80616bdb14d021c4', 'request_id': 'req-ffa05097-3971-498f-9d68-a9e540b50e92'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:47.902779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ffa05097-3971-498f-9d68-a9e540b50e92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db4fb633f434453e80616bdb14d021c4', 'request_id': 'req-ffa05097-3971-498f-9d68-a9e540b50e92'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:47.917604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ffa05097-3971-498f-9d68-a9e540b50e92 admin admin] The token cache is being invalidate because project db4fb633f434453e80616bdb14d021c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:47.917762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ffa05097-3971-498f-9d68-a9e540b50e92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:47.918429 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ffa05097-3971-498f-9d68-a9e540b50e92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:47.920581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1105/2217] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:47 2022] DELETE /identity/v3/projects/db4fb633f434453e80616bdb14d021c4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:47.925149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:47.927750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:47.943270 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb82306-bf4d-4622-83fa-d718b4be645c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:48.024335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:48.025010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:48.025329 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:48.025717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] PATH_INFO: `/v3/users/333f7dac49654c86b910597c157a8572` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:48.026893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=333f7dac49654c86b910597c157a8572)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:48.028925 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:48.090217 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333f7dac49654c86b910597c157a8572'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:48.103729 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333f7dac49654c86b910597c157a8572'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:48.111031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333f7dac49654c86b910597c157a8572'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:48.112295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333f7dac49654c86b910597c157a8572'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:48.112617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333f7dac49654c86b910597c157a8572'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:48.128606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1113/2218] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:47 2022] DELETE /identity/v3/users/333f7dac49654c86b910597c157a8572 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:48.351819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc7bee-33b9-4e7d-acfc-73db64d0c21f None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:48.352769 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc7bee-33b9-4e7d-acfc-73db64d0c21f None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:48.353123 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc7bee-33b9-4e7d-acfc-73db64d0c21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:48.386118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-34bc7bee-33b9-4e7d-acfc-73db64d0c21f None None] MFA Rules not processed for user `d6083ed15f884ffea8550f28addcbfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:48.387472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-34bc7bee-33b9-4e7d-acfc-73db64d0c21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:48.457935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1106/2219] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:48.597467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:48.599286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:48.698175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2a374-2087-4d2f-9bd2-27c7f9d6b657 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:48.703262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad432c-fd4e-4790-9eec-5a76fc788a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:48.703923 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad432c-fd4e-4790-9eec-5a76fc788a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:48.704196 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad432c-fd4e-4790-9eec-5a76fc788a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:48.704469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad432c-fd4e-4790-9eec-5a76fc788a1e admin admin] PATH_INFO: `/v3/projects/22dc65ed9d9d4f46a07f7487eb83d799` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:48.705673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad432c-fd4e-4790-9eec-5a76fc788a1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=22dc65ed9d9d4f46a07f7487eb83d799)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:48.708220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad432c-fd4e-4790-9eec-5a76fc788a1e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:48.740474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:48.771664 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2aad432c-fd4e-4790-9eec-5a76fc788a1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22dc65ed9d9d4f46a07f7487eb83d799', 'request_id': 'req-2aad432c-fd4e-4790-9eec-5a76fc788a1e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:48.778504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2aad432c-fd4e-4790-9eec-5a76fc788a1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22dc65ed9d9d4f46a07f7487eb83d799', 'request_id': 'req-2aad432c-fd4e-4790-9eec-5a76fc788a1e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:48.787268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2aad432c-fd4e-4790-9eec-5a76fc788a1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22dc65ed9d9d4f46a07f7487eb83d799', 'request_id': 'req-2aad432c-fd4e-4790-9eec-5a76fc788a1e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:48.804505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2aad432c-fd4e-4790-9eec-5a76fc788a1e admin admin] The token cache is being invalidate because project 22dc65ed9d9d4f46a07f7487eb83d799 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:48.804802 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2aad432c-fd4e-4790-9eec-5a76fc788a1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:48.805455 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-2aad432c-fd4e-4790-9eec-5a76fc788a1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:48.807268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1114/2220] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:48 2022] DELETE /identity/v3/projects/22dc65ed9d9d4f46a07f7487eb83d799 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:48.813425 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36146042-ccdf-42e6-a078-f761681334da None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:48.814056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36146042-ccdf-42e6-a078-f761681334da None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:48.814534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36146042-ccdf-42e6-a078-f761681334da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:48.841858 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-36146042-ccdf-42e6-a078-f761681334da None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:48.843368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-36146042-ccdf-42e6-a078-f761681334da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:48.921210 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1107/2221] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:48.929943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4db2a-3297-49db-9ce9-ab25063043a7 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:48.930620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4db2a-3297-49db-9ce9-ab25063043a7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:48.932180 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4db2a-3297-49db-9ce9-ab25063043a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:48.955209 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-04a4db2a-3297-49db-9ce9-ab25063043a7 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:48.956396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-04a4db2a-3297-49db-9ce9-ab25063043a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:48.978868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1115/2222] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:48.983405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36146042-ccdf-42e6-a078-f761681334da None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:48.994048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36146042-ccdf-42e6-a078-f761681334da None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:48.998786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae40298-1297-43a3-bf5a-db09f887419f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:48.999447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae40298-1297-43a3-bf5a-db09f887419f admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:48.999725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae40298-1297-43a3-bf5a-db09f887419f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:48.999994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae40298-1297-43a3-bf5a-db09f887419f admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:49.000650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae40298-1297-43a3-bf5a-db09f887419f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:18:49.000952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae40298-1297-43a3-bf5a-db09f887419f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:49.002907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae40298-1297-43a3-bf5a-db09f887419f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:49.016964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1108/2223] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:18:49.022147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4db2a-3297-49db-9ce9-ab25063043a7 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:49.038587 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4db2a-3297-49db-9ce9-ab25063043a7 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:49.045844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85983010-e70e-4fbe-9308-ae874c291400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:49.046728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85983010-e70e-4fbe-9308-ae874c291400 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:49.047173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85983010-e70e-4fbe-9308-ae874c291400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:49.047507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85983010-e70e-4fbe-9308-ae874c291400 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:49.048269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85983010-e70e-4fbe-9308-ae874c291400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:49.050256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85983010-e70e-4fbe-9308-ae874c291400 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:49.119954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1116/2224] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:49 2022] POST /identity/v3/projects => generated 399 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:49.124746 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae40298-1297-43a3-bf5a-db09f887419f admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:49.136313 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae40298-1297-43a3-bf5a-db09f887419f admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:49.141721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:49.253228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f53eb-4615-4372-a304-6cd060850d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:49.254513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f53eb-4615-4372-a304-6cd060850d85 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:49.254894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f53eb-4615-4372-a304-6cd060850d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:49.255897 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f53eb-4615-4372-a304-6cd060850d85 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:49.256730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f53eb-4615-4372-a304-6cd060850d85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:49.258787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f53eb-4615-4372-a304-6cd060850d85 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:49.286844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1109/2225] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:49 2022] POST /identity/v3/users => generated 300 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:49.292311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85983010-e70e-4fbe-9308-ae874c291400 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:49.305227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85983010-e70e-4fbe-9308-ae874c291400 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:49.310920 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a9720-bbc4-40ef-a1ee-0c7f0fdb5ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:49.311668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a9720-bbc4-40ef-a1ee-0c7f0fdb5ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:49.312017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a9720-bbc4-40ef-a1ee-0c7f0fdb5ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:49.312360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a9720-bbc4-40ef-a1ee-0c7f0fdb5ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:49.314390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a9720-bbc4-40ef-a1ee-0c7f0fdb5ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:49.316346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a9720-bbc4-40ef-a1ee-0c7f0fdb5ee9 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:49.337852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1117/2226] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:49.339149 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f53eb-4615-4372-a304-6cd060850d85 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:49.352435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f53eb-4615-4372-a304-6cd060850d85 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:49.359355 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a8a60-12b3-4ebe-bcd4-3e866f82a131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:49.359989 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a8a60-12b3-4ebe-bcd4-3e866f82a131 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:49.360320 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a8a60-12b3-4ebe-bcd4-3e866f82a131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:49.360530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a8a60-12b3-4ebe-bcd4-3e866f82a131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:49.361435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a8a60-12b3-4ebe-bcd4-3e866f82a131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:49.363231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a8a60-12b3-4ebe-bcd4-3e866f82a131 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:49.376905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1110/2227] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:49.382643 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a9720-bbc4-40ef-a1ee-0c7f0fdb5ee9 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:49.396682 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a9720-bbc4-40ef-a1ee-0c7f0fdb5ee9 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:49.402741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6f4b1-56e0-4699-8d02-8b7be779012c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:49.403661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6f4b1-56e0-4699-8d02-8b7be779012c admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:49.405713 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6f4b1-56e0-4699-8d02-8b7be779012c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:49.405713 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6f4b1-56e0-4699-8d02-8b7be779012c admin admin] PATH_INFO: `/v3/projects/48ce994057c44122871e57924c764f49/users/c61592c1bccb4334b76d8cc20b0c9a2d/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:49.428584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6f4b1-56e0-4699-8d02-8b7be779012c admin admin] RBAC: Authorizing `identity:create_grant(project_id=48ce994057c44122871e57924c764f49, user_id=c61592c1bccb4334b76d8cc20b0c9a2d, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:49.432648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6f4b1-56e0-4699-8d02-8b7be779012c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:49.461339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1118/2228] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:49 2022] PUT /identity/v3/projects/48ce994057c44122871e57924c764f49/users/c61592c1bccb4334b76d8cc20b0c9a2d/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:49.464880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3720-da6a-4700-8021-a6747f9db419 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:49.465372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3720-da6a-4700-8021-a6747f9db419 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:49.465802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3720-da6a-4700-8021-a6747f9db419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:49.510629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-7c9b3720-da6a-4700-8021-a6747f9db419 None None] MFA Rules not processed for user `c61592c1bccb4334b76d8cc20b0c9a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:49.513499 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7c9b3720-da6a-4700-8021-a6747f9db419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:49.615282 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1111/2229] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:49 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:49.620202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6f4b1-56e0-4699-8d02-8b7be779012c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:49.698858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6f4b1-56e0-4699-8d02-8b7be779012c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:49.706403 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f12d1-567f-4781-b929-fb969b1d210c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:49.708466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f12d1-567f-4781-b929-fb969b1d210c admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:49.708466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f12d1-567f-4781-b929-fb969b1d210c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:49.708921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f12d1-567f-4781-b929-fb969b1d210c admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:49.710011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f12d1-567f-4781-b929-fb969b1d210c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:49.713620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f12d1-567f-4781-b929-fb969b1d210c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:49.754778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1119/2230] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:49 2022] POST /identity/v3/projects => generated 401 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:49.758847 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3720-da6a-4700-8021-a6747f9db419 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:49.772219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3720-da6a-4700-8021-a6747f9db419 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:49.845482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67096dc2-df34-4fff-817f-1c3b4bc722ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:49.846107 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67096dc2-df34-4fff-817f-1c3b4bc722ac admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:49.846394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67096dc2-df34-4fff-817f-1c3b4bc722ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:49.846687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67096dc2-df34-4fff-817f-1c3b4bc722ac admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:49.847735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67096dc2-df34-4fff-817f-1c3b4bc722ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:49.849811 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67096dc2-df34-4fff-817f-1c3b4bc722ac admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:49.878167 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1112/2231] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:49 2022] POST /identity/v3/users => generated 307 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:49.884119 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f12d1-567f-4781-b929-fb969b1d210c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:49.897876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f12d1-567f-4781-b929-fb969b1d210c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:49.903496 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9eb9b-97ec-4c8c-969a-cdaf8714046a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:49.904219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9eb9b-97ec-4c8c-969a-cdaf8714046a admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:49.904595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9eb9b-97ec-4c8c-969a-cdaf8714046a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:49.904943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9eb9b-97ec-4c8c-969a-cdaf8714046a admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:49.905758 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9eb9b-97ec-4c8c-969a-cdaf8714046a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:49.907573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9eb9b-97ec-4c8c-969a-cdaf8714046a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:49.920710 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1120/2232] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:49.926872 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67096dc2-df34-4fff-817f-1c3b4bc722ac admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:49.934529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:49.942910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67096dc2-df34-4fff-817f-1c3b4bc722ac admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:49.950831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2963e7-8c0f-4f36-aa47-1afa5ac52632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:49.951751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2963e7-8c0f-4f36-aa47-1afa5ac52632 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:49.952203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2963e7-8c0f-4f36-aa47-1afa5ac52632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:49.952621 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2963e7-8c0f-4f36-aa47-1afa5ac52632 admin admin] PATH_INFO: `/v3/projects/b11b8ed895c2401b9d17478286ac1970/users/61a1995e585b4a40bc965489c45378c7/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:49.977625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2963e7-8c0f-4f36-aa47-1afa5ac52632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b11b8ed895c2401b9d17478286ac1970, user_id=61a1995e585b4a40bc965489c45378c7, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:49.981566 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2963e7-8c0f-4f36-aa47-1afa5ac52632 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:49.998511 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1113/2233] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:49 2022] PUT /identity/v3/projects/b11b8ed895c2401b9d17478286ac1970/users/61a1995e585b4a40bc965489c45378c7/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:50.002420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9eb9b-97ec-4c8c-969a-cdaf8714046a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:50.014805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9eb9b-97ec-4c8c-969a-cdaf8714046a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:50.036648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:50.090119 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f28aad-e78a-4ab0-acf1-187959c361f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:50.090950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f28aad-e78a-4ab0-acf1-187959c361f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:50.091278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f28aad-e78a-4ab0-acf1-187959c361f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:50.091554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f28aad-e78a-4ab0-acf1-187959c361f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:50.092379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f28aad-e78a-4ab0-acf1-187959c361f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:50.095250 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f28aad-e78a-4ab0-acf1-187959c361f1 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:50.120309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1121/2234] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:50.122597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2963e7-8c0f-4f36-aa47-1afa5ac52632 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:50.137078 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2963e7-8c0f-4f36-aa47-1afa5ac52632 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:50.145382 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ec81f-07a7-457c-84f0-767e53e516a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:50.147393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ec81f-07a7-457c-84f0-767e53e516a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:50.147903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ec81f-07a7-457c-84f0-767e53e516a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:50.148371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ec81f-07a7-457c-84f0-767e53e516a0 admin admin] PATH_INFO: `/v3/projects/b11b8ed895c2401b9d17478286ac1970/users/61a1995e585b4a40bc965489c45378c7/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:50.150630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ec81f-07a7-457c-84f0-767e53e516a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b11b8ed895c2401b9d17478286ac1970, user_id=61a1995e585b4a40bc965489c45378c7, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:50.153578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ec81f-07a7-457c-84f0-767e53e516a0 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:50.172896 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1114/2235] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:50 2022] PUT /identity/v3/projects/b11b8ed895c2401b9d17478286ac1970/users/61a1995e585b4a40bc965489c45378c7/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:50.185240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d27bc4-bd87-4979-a802-72ffa40666a7 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:50.185534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d27bc4-bd87-4979-a802-72ffa40666a7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:50.185815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d27bc4-bd87-4979-a802-72ffa40666a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:50.221522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-26d27bc4-bd87-4979-a802-72ffa40666a7 None None] MFA Rules not processed for user `61a1995e585b4a40bc965489c45378c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:50.222828 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-26d27bc4-bd87-4979-a802-72ffa40666a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:50.318955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1122/2236] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:50 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:50.330019 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ec81f-07a7-457c-84f0-767e53e516a0 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:50.331839 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-551ec81f-07a7-457c-84f0-767e53e516a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:50.401844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ec81f-07a7-457c-84f0-767e53e516a0 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:50.406828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5c156-a589-4388-9901-b3174b8d75a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:50.407532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5c156-a589-4388-9901-b3174b8d75a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:50.407816 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5c156-a589-4388-9901-b3174b8d75a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:50.408226 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5c156-a589-4388-9901-b3174b8d75a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:50.420679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5c156-a589-4388-9901-b3174b8d75a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:50.422306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5c156-a589-4388-9901-b3174b8d75a9 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:50.434834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1115/2237] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:50 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:50.544281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d27bc4-bd87-4979-a802-72ffa40666a7 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:50.556114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d27bc4-bd87-4979-a802-72ffa40666a7 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:50.559934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89253f2-f06d-4148-ab04-9cfa32828ea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:50.560632 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89253f2-f06d-4148-ab04-9cfa32828ea6 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:50.561008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89253f2-f06d-4148-ab04-9cfa32828ea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:50.561352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89253f2-f06d-4148-ab04-9cfa32828ea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:50.654894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89253f2-f06d-4148-ab04-9cfa32828ea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:50.656747 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89253f2-f06d-4148-ab04-9cfa32828ea6 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:50.671298 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1123/2238] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:50 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:55.021995 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5c156-a589-4388-9901-b3174b8d75a9 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:55.132246 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5c156-a589-4388-9901-b3174b8d75a9 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:55.138779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:55.140183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:55.140798 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:55.141288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] PATH_INFO: `/v3/users/c61592c1bccb4334b76d8cc20b0c9a2d` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:55.142726 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c61592c1bccb4334b76d8cc20b0c9a2d)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:55.145366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:55.211002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c61592c1bccb4334b76d8cc20b0c9a2d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:55.221166 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c61592c1bccb4334b76d8cc20b0c9a2d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:55.222030 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c61592c1bccb4334b76d8cc20b0c9a2d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:55.223268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c61592c1bccb4334b76d8cc20b0c9a2d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:55.245308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c61592c1bccb4334b76d8cc20b0c9a2d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:55.247443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:55.266079 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1116/2239] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:55 2022] DELETE /identity/v3/users/c61592c1bccb4334b76d8cc20b0c9a2d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:55.274548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d92f01-17d0-4e69-b0ba-4f864ad9aaee None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:55.275420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d92f01-17d0-4e69-b0ba-4f864ad9aaee None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:55.275568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d92f01-17d0-4e69-b0ba-4f864ad9aaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:55.306128 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f6d92f01-17d0-4e69-b0ba-4f864ad9aaee None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:55.308410 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f6d92f01-17d0-4e69-b0ba-4f864ad9aaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:55.374824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:55.411881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1124/2240] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:55.423553 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:55.426400 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:55.559477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc7435-eff1-48b9-86bf-38a9b77026ad admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:55.564508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ee8b2-964b-4462-83e7-dd00eed8e49b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:55.565737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ee8b2-964b-4462-83e7-dd00eed8e49b service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:55.567744 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ee8b2-964b-4462-83e7-dd00eed8e49b service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:55.567744 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ee8b2-964b-4462-83e7-dd00eed8e49b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:55.576577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ee8b2-964b-4462-83e7-dd00eed8e49b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:55.578102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ee8b2-964b-4462-83e7-dd00eed8e49b service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:55.593719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1117/2241] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:56.085545 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d92f01-17d0-4e69-b0ba-4f864ad9aaee None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:56.088024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f6d92f01-17d0-4e69-b0ba-4f864ad9aaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:56.107446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d92f01-17d0-4e69-b0ba-4f864ad9aaee None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:56.113824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:56.114490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:56.114766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:56.115391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] PATH_INFO: `/v3/projects/48ce994057c44122871e57924c764f49` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:56.116600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48ce994057c44122871e57924c764f49)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:56.118907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:56.186870 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48ce994057c44122871e57924c764f49', 'request_id': 'req-f933e84c-a257-47f9-b3b7-676eed6fd3b6'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.196294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48ce994057c44122871e57924c764f49', 'request_id': 'req-f933e84c-a257-47f9-b3b7-676eed6fd3b6'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.206337 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48ce994057c44122871e57924c764f49', 'request_id': 'req-f933e84c-a257-47f9-b3b7-676eed6fd3b6'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.241530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] The token cache is being invalidate because project 48ce994057c44122871e57924c764f49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:56.241871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.242615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.244849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1125/2242] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:56 2022] DELETE /identity/v3/projects/48ce994057c44122871e57924c764f49 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:56.249083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ee8b2-964b-4462-83e7-dd00eed8e49b service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:56.260166 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ee8b2-964b-4462-83e7-dd00eed8e49b service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:56.335725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c43ee-f588-4049-b972-8809d35d04e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:56.336732 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c43ee-f588-4049-b972-8809d35d04e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:56.337087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c43ee-f588-4049-b972-8809d35d04e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:56.337466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c43ee-f588-4049-b972-8809d35d04e4 admin admin] PATH_INFO: `/v3/users/61a1995e585b4a40bc965489c45378c7` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:56.338729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c43ee-f588-4049-b972-8809d35d04e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a1995e585b4a40bc965489c45378c7)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:56.340795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c43ee-f588-4049-b972-8809d35d04e4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:56.400543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1f8c43ee-f588-4049-b972-8809d35d04e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a1995e585b4a40bc965489c45378c7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.409527 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1f8c43ee-f588-4049-b972-8809d35d04e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a1995e585b4a40bc965489c45378c7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.410136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1f8c43ee-f588-4049-b972-8809d35d04e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a1995e585b4a40bc965489c45378c7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.410923 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1f8c43ee-f588-4049-b972-8809d35d04e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a1995e585b4a40bc965489c45378c7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.423757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1f8c43ee-f588-4049-b972-8809d35d04e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a1995e585b4a40bc965489c45378c7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:56.436331 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1118/2243] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:56 2022] DELETE /identity/v3/users/61a1995e585b4a40bc965489c45378c7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:57.204529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:57.206121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:57.221719 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933e84c-a257-47f9-b3b7-676eed6fd3b6 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:57.301981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:57.302631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.302914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.303231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0 admin admin] PATH_INFO: `/v3/projects/b11b8ed895c2401b9d17478286ac1970` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.304418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b11b8ed895c2401b9d17478286ac1970)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:57.306474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:57.359829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b11b8ed895c2401b9d17478286ac1970', 'request_id': 'req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:57.366017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b11b8ed895c2401b9d17478286ac1970', 'request_id': 'req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:57.371959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b11b8ed895c2401b9d17478286ac1970', 'request_id': 'req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:57.374512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:57.387595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0 admin admin] The token cache is being invalidate because project b11b8ed895c2401b9d17478286ac1970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:18:57.387804 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:57.388439 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-63b3aa8c-678d-4515-b6be-3c2fbce76bb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:57.389931 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1126/2244] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:57 2022] DELETE /identity/v3/projects/b11b8ed895c2401b9d17478286ac1970 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:57.396038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008317e6-ddaa-44ed-9cc5-bb9c2b26179a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.396608 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008317e6-ddaa-44ed-9cc5-bb9c2b26179a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.396608 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008317e6-ddaa-44ed-9cc5-bb9c2b26179a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.416391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-008317e6-ddaa-44ed-9cc5-bb9c2b26179a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:57.417593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-008317e6-ddaa-44ed-9cc5-bb9c2b26179a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:57.423058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:57.480825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1119/2245] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:18:57.489016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1265407-7c11-4904-a781-ecd49df7ed17 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.489296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1265407-7c11-4904-a781-ecd49df7ed17 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.489567 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1265407-7c11-4904-a781-ecd49df7ed17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.517986 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f1265407-7c11-4904-a781-ecd49df7ed17 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:57.519278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f1265407-7c11-4904-a781-ecd49df7ed17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:57.527619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1127/2246] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:57.532451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008317e6-ddaa-44ed-9cc5-bb9c2b26179a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:57.542909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008317e6-ddaa-44ed-9cc5-bb9c2b26179a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:57.547869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fffda-dd4e-4a3e-8785-658e8645ecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:57.548165 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fffda-dd4e-4a3e-8785-658e8645ecf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.548444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fffda-dd4e-4a3e-8785-658e8645ecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.548717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fffda-dd4e-4a3e-8785-658e8645ecf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.549462 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198fffda-dd4e-4a3e-8785-658e8645ecf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:18:57.549803 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198fffda-dd4e-4a3e-8785-658e8645ecf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:57.552013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198fffda-dd4e-4a3e-8785-658e8645ecf0 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:57.564939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1120/2247] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:18:57.570030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1265407-7c11-4904-a781-ecd49df7ed17 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:57.580866 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1265407-7c11-4904-a781-ecd49df7ed17 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:57.587742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0486b9-1716-42d2-995d-598851892714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:57.588783 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0486b9-1716-42d2-995d-598851892714 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.588783 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0486b9-1716-42d2-995d-598851892714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.589074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0486b9-1716-42d2-995d-598851892714 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.589539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0486b9-1716-42d2-995d-598851892714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:57.591560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0486b9-1716-42d2-995d-598851892714 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:57.627959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1128/2248] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:57 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:57.629938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fffda-dd4e-4a3e-8785-658e8645ecf0 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:57.640669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fffda-dd4e-4a3e-8785-658e8645ecf0 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:57.716457 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10d14a-3a73-4106-adf1-448eaa3be57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:57.717329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10d14a-3a73-4106-adf1-448eaa3be57e admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.717685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10d14a-3a73-4106-adf1-448eaa3be57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.718030 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10d14a-3a73-4106-adf1-448eaa3be57e admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.718817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10d14a-3a73-4106-adf1-448eaa3be57e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:57.720857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10d14a-3a73-4106-adf1-448eaa3be57e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:57.745681 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1121/2249] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:57 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:57.750380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0486b9-1716-42d2-995d-598851892714 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:57.762141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0486b9-1716-42d2-995d-598851892714 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:57.767255 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d8e3a-dd12-4773-aca3-3ef001ec8492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:57.767875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d8e3a-dd12-4773-aca3-3ef001ec8492 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.768159 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d8e3a-dd12-4773-aca3-3ef001ec8492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.768433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d8e3a-dd12-4773-aca3-3ef001ec8492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.769114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d8e3a-dd12-4773-aca3-3ef001ec8492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:57.770756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d8e3a-dd12-4773-aca3-3ef001ec8492 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:57.781670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1129/2250] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:57.786345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10d14a-3a73-4106-adf1-448eaa3be57e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:57.799876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10d14a-3a73-4106-adf1-448eaa3be57e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:57.805549 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e9d12-a651-4177-a24b-5bc945d32829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:57.806172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e9d12-a651-4177-a24b-5bc945d32829 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.806452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e9d12-a651-4177-a24b-5bc945d32829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.806725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e9d12-a651-4177-a24b-5bc945d32829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.807426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e9d12-a651-4177-a24b-5bc945d32829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:57.809039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e9d12-a651-4177-a24b-5bc945d32829 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:57.829223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1122/2251] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:57.833650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d8e3a-dd12-4773-aca3-3ef001ec8492 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:57.844126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d8e3a-dd12-4773-aca3-3ef001ec8492 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:57.849264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736130a-bbed-41b1-88f6-9f2188a704e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:57.849936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736130a-bbed-41b1-88f6-9f2188a704e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.850214 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736130a-bbed-41b1-88f6-9f2188a704e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.850489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736130a-bbed-41b1-88f6-9f2188a704e4 admin admin] PATH_INFO: `/v3/projects/538da930f8b34483bb471945a30aff92/users/5bde94fe22b44e69ad7348db75ef904e/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.869390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736130a-bbed-41b1-88f6-9f2188a704e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538da930f8b34483bb471945a30aff92, user_id=5bde94fe22b44e69ad7348db75ef904e, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:57.872370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736130a-bbed-41b1-88f6-9f2188a704e4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:57.890088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1130/2252] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:57 2022] PUT /identity/v3/projects/538da930f8b34483bb471945a30aff92/users/5bde94fe22b44e69ad7348db75ef904e/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:57.902036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce74cc2-20af-4bf2-91e1-106236612626 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:57.902350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce74cc2-20af-4bf2-91e1-106236612626 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:57.902631 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce74cc2-20af-4bf2-91e1-106236612626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:57.939982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-1ce74cc2-20af-4bf2-91e1-106236612626 None None] MFA Rules not processed for user `5bde94fe22b44e69ad7348db75ef904e`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:57.941252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1ce74cc2-20af-4bf2-91e1-106236612626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:58.027047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1123/2253] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:58.028788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736130a-bbed-41b1-88f6-9f2188a704e4 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.123023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736130a-bbed-41b1-88f6-9f2188a704e4 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.129105 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd409ee3-84c3-4ec2-b199-d5ad8298531e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.130070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd409ee3-84c3-4ec2-b199-d5ad8298531e admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.130528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd409ee3-84c3-4ec2-b199-d5ad8298531e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.130978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd409ee3-84c3-4ec2-b199-d5ad8298531e admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.131875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd409ee3-84c3-4ec2-b199-d5ad8298531e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.134311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd409ee3-84c3-4ec2-b199-d5ad8298531e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.171528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1131/2254] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:18:58 2022] POST /identity/v3/projects => generated 367 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:18:58.172466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce74cc2-20af-4bf2-91e1-106236612626 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.185100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce74cc2-20af-4bf2-91e1-106236612626 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.189838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:58.277477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409946c7-bc5a-413a-aa0c-717acca8eb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.278454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409946c7-bc5a-413a-aa0c-717acca8eb00 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.278732 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409946c7-bc5a-413a-aa0c-717acca8eb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.279082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409946c7-bc5a-413a-aa0c-717acca8eb00 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.279857 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409946c7-bc5a-413a-aa0c-717acca8eb00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.283011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409946c7-bc5a-413a-aa0c-717acca8eb00 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.312101 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1124/2255] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:18:58 2022] POST /identity/v3/users => generated 290 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:18:58.315921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd409ee3-84c3-4ec2-b199-d5ad8298531e admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.329239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd409ee3-84c3-4ec2-b199-d5ad8298531e admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.336951 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5431e7-bf5a-4f4c-9d14-98736e78815d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.338015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5431e7-bf5a-4f4c-9d14-98736e78815d admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.338249 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5431e7-bf5a-4f4c-9d14-98736e78815d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.338628 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5431e7-bf5a-4f4c-9d14-98736e78815d admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.340557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5431e7-bf5a-4f4c-9d14-98736e78815d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.341784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5431e7-bf5a-4f4c-9d14-98736e78815d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.355157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1132/2256] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:18:58.361081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409946c7-bc5a-413a-aa0c-717acca8eb00 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.375601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409946c7-bc5a-413a-aa0c-717acca8eb00 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.383063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84453f-086e-4870-8df5-636d8cac549b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.384018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84453f-086e-4870-8df5-636d8cac549b admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.385985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84453f-086e-4870-8df5-636d8cac549b admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.385985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84453f-086e-4870-8df5-636d8cac549b admin admin] PATH_INFO: `/v3/projects/7ad9c84d699f4b5da927d3527421b8c0/users/c8e69911e272455e9e17430584984558/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.407297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84453f-086e-4870-8df5-636d8cac549b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad9c84d699f4b5da927d3527421b8c0, user_id=c8e69911e272455e9e17430584984558, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.412777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84453f-086e-4870-8df5-636d8cac549b admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.435723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1125/2257] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:58 2022] PUT /identity/v3/projects/7ad9c84d699f4b5da927d3527421b8c0/users/c8e69911e272455e9e17430584984558/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:58.439639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5431e7-bf5a-4f4c-9d14-98736e78815d admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.450063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5431e7-bf5a-4f4c-9d14-98736e78815d admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.524783 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0039e3-5f79-492c-8a72-ae4c917f270a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.525419 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0039e3-5f79-492c-8a72-ae4c917f270a admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.525704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0039e3-5f79-492c-8a72-ae4c917f270a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.525983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0039e3-5f79-492c-8a72-ae4c917f270a admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.526643 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0039e3-5f79-492c-8a72-ae4c917f270a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.528520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0039e3-5f79-492c-8a72-ae4c917f270a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.536025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:58.542701 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1133/2258] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:18:58.546772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84453f-086e-4870-8df5-636d8cac549b admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.559930 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84453f-086e-4870-8df5-636d8cac549b admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.564924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1ae80-620b-47d9-be46-22cca6dbfbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.565752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1ae80-620b-47d9-be46-22cca6dbfbb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.566092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1ae80-620b-47d9-be46-22cca6dbfbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.566456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1ae80-620b-47d9-be46-22cca6dbfbb2 admin admin] PATH_INFO: `/v3/projects/7ad9c84d699f4b5da927d3527421b8c0/users/c8e69911e272455e9e17430584984558/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.568545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1ae80-620b-47d9-be46-22cca6dbfbb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad9c84d699f4b5da927d3527421b8c0, user_id=c8e69911e272455e9e17430584984558, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.571606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1ae80-620b-47d9-be46-22cca6dbfbb2 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.592581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1126/2259] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:18:58 2022] PUT /identity/v3/projects/7ad9c84d699f4b5da927d3527421b8c0/users/c8e69911e272455e9e17430584984558/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:18:58.604756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ad4e6-6920-4c7a-992a-a89bb2d100e0 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.605046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ad4e6-6920-4c7a-992a-a89bb2d100e0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.605313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ad4e6-6920-4c7a-992a-a89bb2d100e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.647657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-1a4ad4e6-6920-4c7a-992a-a89bb2d100e0 None None] MFA Rules not processed for user `c8e69911e272455e9e17430584984558`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:58.649631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1a4ad4e6-6920-4c7a-992a-a89bb2d100e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:58.738150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1134/2260] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:58.751665 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1ae80-620b-47d9-be46-22cca6dbfbb2 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.754198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-fdc1ae80-620b-47d9-be46-22cca6dbfbb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:58.819703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1ae80-620b-47d9-be46-22cca6dbfbb2 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.823749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ddc6d-b44c-4a3b-9a25-bf6c6282b5db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.824505 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ddc6d-b44c-4a3b-9a25-bf6c6282b5db service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.824775 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ddc6d-b44c-4a3b-9a25-bf6c6282b5db service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.825113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ddc6d-b44c-4a3b-9a25-bf6c6282b5db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:58.836071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ddc6d-b44c-4a3b-9a25-bf6c6282b5db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:58.837536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ddc6d-b44c-4a3b-9a25-bf6c6282b5db service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:58.848190 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1127/2261] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:58.955436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ad4e6-6920-4c7a-992a-a89bb2d100e0 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:58.974313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ad4e6-6920-4c7a-992a-a89bb2d100e0 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:58.979408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aedffa-e77c-414d-a481-48450a69a031 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:58.980199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aedffa-e77c-414d-a481-48450a69a031 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:58.980880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aedffa-e77c-414d-a481-48450a69a031 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:58.981324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aedffa-e77c-414d-a481-48450a69a031 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:59.046480 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aedffa-e77c-414d-a481-48450a69a031 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:59.048196 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aedffa-e77c-414d-a481-48450a69a031 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:59.059734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1135/2262] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:59.093176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ddc6d-b44c-4a3b-9a25-bf6c6282b5db service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:59.095568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-309ddc6d-b44c-4a3b-9a25-bf6c6282b5db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:59.173138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ddc6d-b44c-4a3b-9a25-bf6c6282b5db service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:59.179471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:59.180161 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:59.180433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:59.180728 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] PATH_INFO: `/v3/users/d6ae71f1deb947379fa12f0c8f4ee02f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:59.181971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6ae71f1deb947379fa12f0c8f4ee02f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:59.184103 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:59.187002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:18:59.264189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6ae71f1deb947379fa12f0c8f4ee02f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:59.281587 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6ae71f1deb947379fa12f0c8f4ee02f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:59.282752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6ae71f1deb947379fa12f0c8f4ee02f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:59.284115 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6ae71f1deb947379fa12f0c8f4ee02f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:59.311564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6ae71f1deb947379fa12f0c8f4ee02f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:18:59.338923 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1128/2263] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:18:59 2022] DELETE /identity/v3/users/d6ae71f1deb947379fa12f0c8f4ee02f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:18:59.346922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e27428-3a3c-49ba-92ca-76a3425f9178 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:59.348327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e27428-3a3c-49ba-92ca-76a3425f9178 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:59.348512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e27428-3a3c-49ba-92ca-76a3425f9178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:59.383092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-74e27428-3a3c-49ba-92ca-76a3425f9178 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:18:59.385178 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-74e27428-3a3c-49ba-92ca-76a3425f9178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:59.496986 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1136/2264] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:18:59.509670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:59.511544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:18:59.588945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665c742-f369-432d-94a7-c4b1ee060976 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:18:59.593139 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b9e5b-4d8e-49c8-abe0-4b899cd11797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:18:59.593301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b9e5b-4d8e-49c8-abe0-4b899cd11797 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:18:59.593477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b9e5b-4d8e-49c8-abe0-4b899cd11797 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:18:59.594700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b9e5b-4d8e-49c8-abe0-4b899cd11797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:18:59.604461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b9e5b-4d8e-49c8-abe0-4b899cd11797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:18:59.606567 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b9e5b-4d8e-49c8-abe0-4b899cd11797 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:18:59.619205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1129/2265] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:18:59.989647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e27428-3a3c-49ba-92ca-76a3425f9178 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:18:59.991040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-74e27428-3a3c-49ba-92ca-76a3425f9178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:00.006872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e27428-3a3c-49ba-92ca-76a3425f9178 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:00.012385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:00.013063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.013345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.013650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c admin admin] PATH_INFO: `/v3/projects/7a1b904157204964b4c04d0772627052` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.014808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1b904157204964b4c04d0772627052)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:00.016771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:00.042615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=100 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:00.076352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1b904157204964b4c04d0772627052', 'request_id': 'req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:00.083584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1b904157204964b4c04d0772627052', 'request_id': 'req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:00.090046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1b904157204964b4c04d0772627052', 'request_id': 'req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:00.104853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c admin admin] The token cache is being invalidate because project 7a1b904157204964b4c04d0772627052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:00.105147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:00.106054 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-e17cc888-db30-4b92-b3b4-d9855ecc3a5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:00.107631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1137/2266] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:18:59 2022] DELETE /identity/v3/projects/7a1b904157204964b4c04d0772627052 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:00.112482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6dd55-456f-47e1-92da-6625672385a9 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.112838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6dd55-456f-47e1-92da-6625672385a9 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.113188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6dd55-456f-47e1-92da-6625672385a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.141119 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-e9a6dd55-456f-47e1-92da-6625672385a9 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:00.143203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e9a6dd55-456f-47e1-92da-6625672385a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:00.217641 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1130/2267] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:00.225652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2576e0a-7950-4b79-8455-b5817696284e None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.225963 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2576e0a-7950-4b79-8455-b5817696284e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.226244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2576e0a-7950-4b79-8455-b5817696284e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.248402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-c2576e0a-7950-4b79-8455-b5817696284e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:00.249917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c2576e0a-7950-4b79-8455-b5817696284e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:00.258622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1138/2268] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:00.262889 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6dd55-456f-47e1-92da-6625672385a9 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:00.273862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6dd55-456f-47e1-92da-6625672385a9 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:00.278544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33279663-16e6-4bc4-8c23-e48586f2db20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:00.279181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33279663-16e6-4bc4-8c23-e48586f2db20 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.279518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33279663-16e6-4bc4-8c23-e48586f2db20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.279795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33279663-16e6-4bc4-8c23-e48586f2db20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.280447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33279663-16e6-4bc4-8c23-e48586f2db20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:00.280753 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33279663-16e6-4bc4-8c23-e48586f2db20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:00.282376 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33279663-16e6-4bc4-8c23-e48586f2db20 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:00.300161 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1131/2269] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:00.301409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2576e0a-7950-4b79-8455-b5817696284e None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:00.311180 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2576e0a-7950-4b79-8455-b5817696284e None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:00.317766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c3ff9-0981-4114-8a17-69e54a3ea69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:00.318343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c3ff9-0981-4114-8a17-69e54a3ea69b admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.318626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c3ff9-0981-4114-8a17-69e54a3ea69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.318900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c3ff9-0981-4114-8a17-69e54a3ea69b admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.319760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c3ff9-0981-4114-8a17-69e54a3ea69b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:00.321706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c3ff9-0981-4114-8a17-69e54a3ea69b admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:00.356597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1139/2270] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:00 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:00.357747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33279663-16e6-4bc4-8c23-e48586f2db20 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:00.368024 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33279663-16e6-4bc4-8c23-e48586f2db20 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:00.376806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:00.433695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005b621-ea86-493f-8278-94651d34e195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:00.434306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005b621-ea86-493f-8278-94651d34e195 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.434583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005b621-ea86-493f-8278-94651d34e195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.434846 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005b621-ea86-493f-8278-94651d34e195 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.435606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005b621-ea86-493f-8278-94651d34e195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:00.437821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005b621-ea86-493f-8278-94651d34e195 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:00.468181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1132/2271] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:00 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:00.472552 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c3ff9-0981-4114-8a17-69e54a3ea69b admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:00.484595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c3ff9-0981-4114-8a17-69e54a3ea69b admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:00.491706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90346c9-20fd-472f-917f-c5228afc89d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:00.492436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90346c9-20fd-472f-917f-c5228afc89d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.492780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90346c9-20fd-472f-917f-c5228afc89d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.493111 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90346c9-20fd-472f-917f-c5228afc89d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.493850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90346c9-20fd-472f-917f-c5228afc89d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:00.495691 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90346c9-20fd-472f-917f-c5228afc89d5 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:00.511995 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1140/2272] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:00.514756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005b621-ea86-493f-8278-94651d34e195 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:00.526540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005b621-ea86-493f-8278-94651d34e195 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:00.531783 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb9dd9-ca48-4dd6-870f-75f54e96b099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:00.532385 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb9dd9-ca48-4dd6-870f-75f54e96b099 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.532669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb9dd9-ca48-4dd6-870f-75f54e96b099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.532946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb9dd9-ca48-4dd6-870f-75f54e96b099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.533611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb9dd9-ca48-4dd6-870f-75f54e96b099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:00.535440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb9dd9-ca48-4dd6-870f-75f54e96b099 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:00.550434 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1133/2273] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:00.558563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90346c9-20fd-472f-917f-c5228afc89d5 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:00.579029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90346c9-20fd-472f-917f-c5228afc89d5 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:00.586358 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a157c9f-b88e-4f89-a1e4-94d22bf6dcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:00.587039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a157c9f-b88e-4f89-a1e4-94d22bf6dcf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.587357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a157c9f-b88e-4f89-a1e4-94d22bf6dcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.587638 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a157c9f-b88e-4f89-a1e4-94d22bf6dcf9 admin admin] PATH_INFO: `/v3/projects/4df9cdf40a374879bb268ad198cad5dd/users/d23e497861da45c79e990490d1ebcb89/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.610084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a157c9f-b88e-4f89-a1e4-94d22bf6dcf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4df9cdf40a374879bb268ad198cad5dd, user_id=d23e497861da45c79e990490d1ebcb89, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:00.614228 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a157c9f-b88e-4f89-a1e4-94d22bf6dcf9 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:00.633860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1141/2274] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:00 2022] PUT /identity/v3/projects/4df9cdf40a374879bb268ad198cad5dd/users/d23e497861da45c79e990490d1ebcb89/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:00.645473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37094572-f8bc-4067-9531-1366547a747f None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.645778 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37094572-f8bc-4067-9531-1366547a747f None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.646055 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37094572-f8bc-4067-9531-1366547a747f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.677748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-37094572-f8bc-4067-9531-1366547a747f None None] MFA Rules not processed for user `d23e497861da45c79e990490d1ebcb89`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:00.678985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-37094572-f8bc-4067-9531-1366547a747f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:00.758794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1134/2275] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:00.763303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a157c9f-b88e-4f89-a1e4-94d22bf6dcf9 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:00.890895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a157c9f-b88e-4f89-a1e4-94d22bf6dcf9 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:00.900651 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61636b81-52c2-4b94-9619-8bfd99dc2b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:00.901623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61636b81-52c2-4b94-9619-8bfd99dc2b9b admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:00.902079 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61636b81-52c2-4b94-9619-8bfd99dc2b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:00.902509 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61636b81-52c2-4b94-9619-8bfd99dc2b9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:00.903397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61636b81-52c2-4b94-9619-8bfd99dc2b9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:00.905362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61636b81-52c2-4b94-9619-8bfd99dc2b9b admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:00.943983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1142/2276] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:00 2022] POST /identity/v3/projects => generated 373 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:00.948975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37094572-f8bc-4067-9531-1366547a747f None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:00.962144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37094572-f8bc-4067-9531-1366547a747f None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:01.037204 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce744303-85c0-411e-94f5-e3a5e39244a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:01.037947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce744303-85c0-411e-94f5-e3a5e39244a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:01.038291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce744303-85c0-411e-94f5-e3a5e39244a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:01.038634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce744303-85c0-411e-94f5-e3a5e39244a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:01.039388 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce744303-85c0-411e-94f5-e3a5e39244a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:01.041526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce744303-85c0-411e-94f5-e3a5e39244a6 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:01.077004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1135/2277] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:00 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:01.081028 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61636b81-52c2-4b94-9619-8bfd99dc2b9b admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:01.104303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61636b81-52c2-4b94-9619-8bfd99dc2b9b admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:01.111695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146a367-dbae-4890-a104-540b1f8e206f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:01.112936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146a367-dbae-4890-a104-540b1f8e206f admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:01.112936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146a367-dbae-4890-a104-540b1f8e206f admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:01.113149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146a367-dbae-4890-a104-540b1f8e206f admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:01.113935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146a367-dbae-4890-a104-540b1f8e206f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:01.115730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146a367-dbae-4890-a104-540b1f8e206f admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:01.129247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1143/2278] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:01.134688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce744303-85c0-411e-94f5-e3a5e39244a6 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:01.156538 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce744303-85c0-411e-94f5-e3a5e39244a6 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:01.166730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ae77e-b023-438c-816e-2fa20607fbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:01.168256 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ae77e-b023-438c-816e-2fa20607fbf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:01.168983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ae77e-b023-438c-816e-2fa20607fbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:01.169368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ae77e-b023-438c-816e-2fa20607fbf5 admin admin] PATH_INFO: `/v3/projects/2e41222b022b44a9ba72a07103ae475d/users/1abdc6b13682477fb767052b292a8b04/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:01.174616 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:01.209101 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ae77e-b023-438c-816e-2fa20607fbf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e41222b022b44a9ba72a07103ae475d, user_id=1abdc6b13682477fb767052b292a8b04, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:01.215677 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ae77e-b023-438c-816e-2fa20607fbf5 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:01.242370 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1136/2279] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:01 2022] PUT /identity/v3/projects/2e41222b022b44a9ba72a07103ae475d/users/1abdc6b13682477fb767052b292a8b04/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:01.248499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146a367-dbae-4890-a104-540b1f8e206f admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:01.266875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146a367-dbae-4890-a104-540b1f8e206f admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:01.282676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:01.337947 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20309373-cbb2-411c-be11-4e9441d0e5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:01.338487 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20309373-cbb2-411c-be11-4e9441d0e5ff admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:01.338772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20309373-cbb2-411c-be11-4e9441d0e5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:01.339047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20309373-cbb2-411c-be11-4e9441d0e5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:01.339753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20309373-cbb2-411c-be11-4e9441d0e5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:01.341379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20309373-cbb2-411c-be11-4e9441d0e5ff admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:01.353773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1144/2280] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:01.358442 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ae77e-b023-438c-816e-2fa20607fbf5 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:01.374981 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ae77e-b023-438c-816e-2fa20607fbf5 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:01.380017 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b145c4-6ac2-4c09-8e0d-cdf10d7f5b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:01.380722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b145c4-6ac2-4c09-8e0d-cdf10d7f5b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:01.380997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b145c4-6ac2-4c09-8e0d-cdf10d7f5b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:01.381268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b145c4-6ac2-4c09-8e0d-cdf10d7f5b41 admin admin] PATH_INFO: `/v3/projects/2e41222b022b44a9ba72a07103ae475d/users/1abdc6b13682477fb767052b292a8b04/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:01.383378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b145c4-6ac2-4c09-8e0d-cdf10d7f5b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e41222b022b44a9ba72a07103ae475d, user_id=1abdc6b13682477fb767052b292a8b04, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:01.386314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b145c4-6ac2-4c09-8e0d-cdf10d7f5b41 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:01.398971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1137/2281] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:01 2022] PUT /identity/v3/projects/2e41222b022b44a9ba72a07103ae475d/users/1abdc6b13682477fb767052b292a8b04/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:01.417590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91389c1e-2445-4395-9f63-1755ec038ed0 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:01.418377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91389c1e-2445-4395-9f63-1755ec038ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:01.418377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91389c1e-2445-4395-9f63-1755ec038ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:01.448763 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-91389c1e-2445-4395-9f63-1755ec038ed0 None None] MFA Rules not processed for user `1abdc6b13682477fb767052b292a8b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:01.450018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-91389c1e-2445-4395-9f63-1755ec038ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:01.526506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1145/2282] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:01.539306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b145c4-6ac2-4c09-8e0d-cdf10d7f5b41 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:01.541208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-30b145c4-6ac2-4c09-8e0d-cdf10d7f5b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:01.604940 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b145c4-6ac2-4c09-8e0d-cdf10d7f5b41 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:01.608767 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cb97a-524b-4308-b341-e2761908c78b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:01.609381 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cb97a-524b-4308-b341-e2761908c78b service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:01.609656 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cb97a-524b-4308-b341-e2761908c78b service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:01.609921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cb97a-524b-4308-b341-e2761908c78b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:01.668127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cb97a-524b-4308-b341-e2761908c78b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:01.669579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cb97a-524b-4308-b341-e2761908c78b service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:01.680046 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1138/2283] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:04.257075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91389c1e-2445-4395-9f63-1755ec038ed0 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:04.258705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-91389c1e-2445-4395-9f63-1755ec038ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:04.333537 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91389c1e-2445-4395-9f63-1755ec038ed0 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:04.338339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b196e-cfd6-49f9-9cb3-93c3a9a4acbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:04.338981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b196e-cfd6-49f9-9cb3-93c3a9a4acbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.339359 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b196e-cfd6-49f9-9cb3-93c3a9a4acbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.339639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b196e-cfd6-49f9-9cb3-93c3a9a4acbb admin admin] PATH_INFO: `/v3/users/5bde94fe22b44e69ad7348db75ef904e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.340837 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b196e-cfd6-49f9-9cb3-93c3a9a4acbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bde94fe22b44e69ad7348db75ef904e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:04.342800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b196e-cfd6-49f9-9cb3-93c3a9a4acbb admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:04.379862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cb97a-524b-4308-b341-e2761908c78b service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:04.393368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cb97a-524b-4308-b341-e2761908c78b service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:04.397959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:04.398338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7730aa-21f4-41d4-8051-3c2e1588c37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:04.399086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7730aa-21f4-41d4-8051-3c2e1588c37e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.399397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7730aa-21f4-41d4-8051-3c2e1588c37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.399679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7730aa-21f4-41d4-8051-3c2e1588c37e admin admin] PATH_INFO: `/v3/users/d23e497861da45c79e990490d1ebcb89` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.400912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7730aa-21f4-41d4-8051-3c2e1588c37e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23e497861da45c79e990490d1ebcb89)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:04.402922 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7730aa-21f4-41d4-8051-3c2e1588c37e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:04.405971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-be5b196e-cfd6-49f9-9cb3-93c3a9a4acbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bde94fe22b44e69ad7348db75ef904e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.420630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-be5b196e-cfd6-49f9-9cb3-93c3a9a4acbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bde94fe22b44e69ad7348db75ef904e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.428813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-be5b196e-cfd6-49f9-9cb3-93c3a9a4acbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bde94fe22b44e69ad7348db75ef904e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.429539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-be5b196e-cfd6-49f9-9cb3-93c3a9a4acbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bde94fe22b44e69ad7348db75ef904e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.430163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-be5b196e-cfd6-49f9-9cb3-93c3a9a4acbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bde94fe22b44e69ad7348db75ef904e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.444410 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1146/2284] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:04 2022] DELETE /identity/v3/users/5bde94fe22b44e69ad7348db75ef904e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:04.449337 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b54d8f-e822-4a3f-8886-784fa71de3b4 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.449648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b54d8f-e822-4a3f-8886-784fa71de3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.449945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b54d8f-e822-4a3f-8886-784fa71de3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.456770 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ec7730aa-21f4-41d4-8051-3c2e1588c37e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23e497861da45c79e990490d1ebcb89'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.470024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-d7b54d8f-e822-4a3f-8886-784fa71de3b4 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:04.471272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d7b54d8f-e822-4a3f-8886-784fa71de3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:04.473532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ec7730aa-21f4-41d4-8051-3c2e1588c37e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23e497861da45c79e990490d1ebcb89'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.474241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ec7730aa-21f4-41d4-8051-3c2e1588c37e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23e497861da45c79e990490d1ebcb89'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.474967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ec7730aa-21f4-41d4-8051-3c2e1588c37e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23e497861da45c79e990490d1ebcb89'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.493737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ec7730aa-21f4-41d4-8051-3c2e1588c37e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23e497861da45c79e990490d1ebcb89'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:04.506633 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:04.517153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1139/2285] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:04 2022] DELETE /identity/v3/users/d23e497861da45c79e990490d1ebcb89 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:04.525154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0f2d6-cd6a-4216-9f6c-1f45e34b817a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.526022 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0f2d6-cd6a-4216-9f6c-1f45e34b817a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.526468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0f2d6-cd6a-4216-9f6c-1f45e34b817a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.548388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1147/2286] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:04.553926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-cfe0f2d6-cd6a-4216-9f6c-1f45e34b817a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:04.555187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-cfe0f2d6-cd6a-4216-9f6c-1f45e34b817a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:04.560115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b54d8f-e822-4a3f-8886-784fa71de3b4 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:04.561646 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d7b54d8f-e822-4a3f-8886-784fa71de3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:04.641301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b54d8f-e822-4a3f-8886-784fa71de3b4 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:04.647626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1140/2287] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:04.659785 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0f2d6-cd6a-4216-9f6c-1f45e34b817a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:04.661138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-cfe0f2d6-cd6a-4216-9f6c-1f45e34b817a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:04.709591 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f533e7-777b-4468-af47-909014f41596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:04.710239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f533e7-777b-4468-af47-909014f41596 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.710521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f533e7-777b-4468-af47-909014f41596 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.710798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f533e7-777b-4468-af47-909014f41596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.721009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f533e7-777b-4468-af47-909014f41596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:04.723204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f533e7-777b-4468-af47-909014f41596 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:04.733304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0f2d6-cd6a-4216-9f6c-1f45e34b817a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:04.735544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1148/2288] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:04.737994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2345d-6800-4ef3-be43-78201f184ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:04.738825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2345d-6800-4ef3-be43-78201f184ffc service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:04.738988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2345d-6800-4ef3-be43-78201f184ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:04.739169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2345d-6800-4ef3-be43-78201f184ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:04.751006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2345d-6800-4ef3-be43-78201f184ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:04.752545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2345d-6800-4ef3-be43-78201f184ffc service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:04.767081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1141/2289] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:05.072021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f533e7-777b-4468-af47-909014f41596 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:05.072313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-60f533e7-777b-4468-af47-909014f41596 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:05.089146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f533e7-777b-4468-af47-909014f41596 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:05.094929 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:05.096157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:05.096849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:05.097346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] PATH_INFO: `/v3/projects/538da930f8b34483bb471945a30aff92` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:05.098872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] RBAC: Authorizing `identity:delete_project(project_id=538da930f8b34483bb471945a30aff92)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:05.101342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:05.115521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:05.161506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '538da930f8b34483bb471945a30aff92', 'request_id': 'req-23cceb74-493f-4c1c-9534-c85bcfcb301c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.168977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '538da930f8b34483bb471945a30aff92', 'request_id': 'req-23cceb74-493f-4c1c-9534-c85bcfcb301c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.175853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '538da930f8b34483bb471945a30aff92', 'request_id': 'req-23cceb74-493f-4c1c-9534-c85bcfcb301c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.181558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2345d-6800-4ef3-be43-78201f184ffc service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:05.183519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b2e2345d-6800-4ef3-be43-78201f184ffc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:05.192858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] The token cache is being invalidate because project 538da930f8b34483bb471945a30aff92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:05.193142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.194017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.199749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1149/2290] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:05 2022] DELETE /identity/v3/projects/538da930f8b34483bb471945a30aff92 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:05.201183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:05.203133 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:05.206528 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2345d-6800-4ef3-be43-78201f184ffc service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:05.270561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:05.271234 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:05.271516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:05.271789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] PATH_INFO: `/v3/projects/4df9cdf40a374879bb268ad198cad5dd` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:05.272931 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4df9cdf40a374879bb268ad198cad5dd)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:05.274831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:05.289253 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cceb74-493f-4c1c-9534-c85bcfcb301c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:05.294941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:05.295775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:05.296193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:05.296551 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] PATH_INFO: `/v3/users/c8e69911e272455e9e17430584984558` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:05.298168 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e69911e272455e9e17430584984558)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:05.300391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:05.332277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:05.338368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4df9cdf40a374879bb268ad198cad5dd', 'request_id': 'req-ba6d848e-4908-4f3f-8241-87976154775a'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.348026 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4df9cdf40a374879bb268ad198cad5dd', 'request_id': 'req-ba6d848e-4908-4f3f-8241-87976154775a'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.355543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4df9cdf40a374879bb268ad198cad5dd', 'request_id': 'req-ba6d848e-4908-4f3f-8241-87976154775a'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.361372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e69911e272455e9e17430584984558'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.370157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] The token cache is being invalidate because project 4df9cdf40a374879bb268ad198cad5dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:05.370455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.371531 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.373879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1142/2291] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:05 2022] DELETE /identity/v3/projects/4df9cdf40a374879bb268ad198cad5dd => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:05.378665 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:05.380714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e69911e272455e9e17430584984558'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.384353 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:05.390768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e69911e272455e9e17430584984558'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.391806 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e69911e272455e9e17430584984558'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.395543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e69911e272455e9e17430584984558'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.419342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1150/2292] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:05 2022] DELETE /identity/v3/users/c8e69911e272455e9e17430584984558 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:05.462005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d848e-4908-4f3f-8241-87976154775a admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:05.528591 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b62837-a99e-4f67-ba82-77fa38e1ee6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:05.529257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b62837-a99e-4f67-ba82-77fa38e1ee6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:05.529567 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b62837-a99e-4f67-ba82-77fa38e1ee6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:05.529854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b62837-a99e-4f67-ba82-77fa38e1ee6a admin admin] PATH_INFO: `/v3/users/1abdc6b13682477fb767052b292a8b04` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:05.531137 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b62837-a99e-4f67-ba82-77fa38e1ee6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1abdc6b13682477fb767052b292a8b04)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:05.533066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b62837-a99e-4f67-ba82-77fa38e1ee6a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:05.599491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f3b62837-a99e-4f67-ba82-77fa38e1ee6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1abdc6b13682477fb767052b292a8b04'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.609216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f3b62837-a99e-4f67-ba82-77fa38e1ee6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1abdc6b13682477fb767052b292a8b04'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.609869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f3b62837-a99e-4f67-ba82-77fa38e1ee6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1abdc6b13682477fb767052b292a8b04'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.610512 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f3b62837-a99e-4f67-ba82-77fa38e1ee6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1abdc6b13682477fb767052b292a8b04'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.627765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f3b62837-a99e-4f67-ba82-77fa38e1ee6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1abdc6b13682477fb767052b292a8b04'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.637306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:05.644789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1143/2293] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:05 2022] DELETE /identity/v3/users/1abdc6b13682477fb767052b292a8b04 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:05.724348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:05.725864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:05.745882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f439c-5a74-4721-86a1-bd04ea4b77eb admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:05.778690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:05.843049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:05.844122 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:05.844567 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:05.845450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] PATH_INFO: `/v3/projects/7ad9c84d699f4b5da927d3527421b8c0` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:05.848104 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ad9c84d699f4b5da927d3527421b8c0)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:05.850876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:05.914045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ad9c84d699f4b5da927d3527421b8c0', 'request_id': 'req-5ca886d6-31d5-489b-90c3-014739a5131a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.921018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ad9c84d699f4b5da927d3527421b8c0', 'request_id': 'req-5ca886d6-31d5-489b-90c3-014739a5131a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.927330 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ad9c84d699f4b5da927d3527421b8c0', 'request_id': 'req-5ca886d6-31d5-489b-90c3-014739a5131a'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.941951 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] The token cache is being invalidate because project 7ad9c84d699f4b5da927d3527421b8c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:05.942246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.942913 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:05.944739 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1151/2294] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:05 2022] DELETE /identity/v3/projects/7ad9c84d699f4b5da927d3527421b8c0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:05.949547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359354aa-5fc6-428c-827c-9e7d4e84d207 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:05.949844 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359354aa-5fc6-428c-827c-9e7d4e84d207 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:05.950127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359354aa-5fc6-428c-827c-9e7d4e84d207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:05.972269 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-359354aa-5fc6-428c-827c-9e7d4e84d207 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:05.973503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-359354aa-5fc6-428c-827c-9e7d4e84d207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:06.014930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.016631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:06.046195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1144/2295] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:06.054789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d55991-2f8c-4bd1-b714-f49d76a93f78 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.055087 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d55991-2f8c-4bd1-b714-f49d76a93f78 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.055373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d55991-2f8c-4bd1-b714-f49d76a93f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.077323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f0d55991-2f8c-4bd1-b714-f49d76a93f78 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:06.078547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f0d55991-2f8c-4bd1-b714-f49d76a93f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:06.093172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1145/2296] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:06.098373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d55991-2f8c-4bd1-b714-f49d76a93f78 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.109059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d55991-2f8c-4bd1-b714-f49d76a93f78 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.114361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca187ab-5034-4ce6-b214-fb3c881dcb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.114979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca187ab-5034-4ce6-b214-fb3c881dcb8c admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.115313 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca187ab-5034-4ce6-b214-fb3c881dcb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.115602 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca187ab-5034-4ce6-b214-fb3c881dcb8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.116242 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca187ab-5034-4ce6-b214-fb3c881dcb8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:06.116543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca187ab-5034-4ce6-b214-fb3c881dcb8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.118215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca187ab-5034-4ce6-b214-fb3c881dcb8c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.130988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca886d6-31d5-489b-90c3-014739a5131a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.135211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1146/2297] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:19:06.136725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca187ab-5034-4ce6-b214-fb3c881dcb8c admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.140279 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170325bb-3f10-40b7-893b-fdd6cc468ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.141186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170325bb-3f10-40b7-893b-fdd6cc468ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.141472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170325bb-3f10-40b7-893b-fdd6cc468ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.141754 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170325bb-3f10-40b7-893b-fdd6cc468ce1 admin admin] PATH_INFO: `/v3/projects/2e41222b022b44a9ba72a07103ae475d` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.142935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170325bb-3f10-40b7-893b-fdd6cc468ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e41222b022b44a9ba72a07103ae475d)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.145512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170325bb-3f10-40b7-893b-fdd6cc468ce1 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.149015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca187ab-5034-4ce6-b214-fb3c881dcb8c admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.155291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b73de6-3df2-4824-974a-d65426799eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.156448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b73de6-3df2-4824-974a-d65426799eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.156733 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b73de6-3df2-4824-974a-d65426799eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.157009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b73de6-3df2-4824-974a-d65426799eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.157690 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b73de6-3df2-4824-974a-d65426799eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.160237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b73de6-3df2-4824-974a-d65426799eb8 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.196482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1147/2298] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:06 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:06.200967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b73de6-3df2-4824-974a-d65426799eb8 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.213904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-170325bb-3f10-40b7-893b-fdd6cc468ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e41222b022b44a9ba72a07103ae475d', 'request_id': 'req-170325bb-3f10-40b7-893b-fdd6cc468ce1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:06.216017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:06.222471 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-170325bb-3f10-40b7-893b-fdd6cc468ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e41222b022b44a9ba72a07103ae475d', 'request_id': 'req-170325bb-3f10-40b7-893b-fdd6cc468ce1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:06.233404 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-170325bb-3f10-40b7-893b-fdd6cc468ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e41222b022b44a9ba72a07103ae475d', 'request_id': 'req-170325bb-3f10-40b7-893b-fdd6cc468ce1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:06.251188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-170325bb-3f10-40b7-893b-fdd6cc468ce1 admin admin] The token cache is being invalidate because project 2e41222b022b44a9ba72a07103ae475d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:06.251525 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-170325bb-3f10-40b7-893b-fdd6cc468ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:06.252302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-170325bb-3f10-40b7-893b-fdd6cc468ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:06.257637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1152/2299] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:06 2022] DELETE /identity/v3/projects/2e41222b022b44a9ba72a07103ae475d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:06.259817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5696c21-6053-4f63-8fd1-fce51da4ff23 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.260111 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5696c21-6053-4f63-8fd1-fce51da4ff23 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.260433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5696c21-6053-4f63-8fd1-fce51da4ff23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.282778 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-d5696c21-6053-4f63-8fd1-fce51da4ff23 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:06.283036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b73de6-3df2-4824-974a-d65426799eb8 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.284631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d5696c21-6053-4f63-8fd1-fce51da4ff23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:06.361819 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1153/2300] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:06.362506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cf3f8-faf6-416a-863b-b4e59bfc0142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.362506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cf3f8-faf6-416a-863b-b4e59bfc0142 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.362506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cf3f8-faf6-416a-863b-b4e59bfc0142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.362506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cf3f8-faf6-416a-863b-b4e59bfc0142 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.362935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06cf3f8-faf6-416a-863b-b4e59bfc0142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.362935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06cf3f8-faf6-416a-863b-b4e59bfc0142 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.364104 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe0e39-fd34-4f1a-9018-578195b474c6 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.364605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe0e39-fd34-4f1a-9018-578195b474c6 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.365021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe0e39-fd34-4f1a-9018-578195b474c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.373988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:06.391260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1148/2301] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:06 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:06.394111 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cf3f8-faf6-416a-863b-b4e59bfc0142 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.394372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-38fe0e39-fd34-4f1a-9018-578195b474c6 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:06.395771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-38fe0e39-fd34-4f1a-9018-578195b474c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:06.396370 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b06cf3f8-faf6-416a-863b-b4e59bfc0142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:06.405720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1154/2302] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:06.410506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe0e39-fd34-4f1a-9018-578195b474c6 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.414011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cf3f8-faf6-416a-863b-b4e59bfc0142 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.420464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79fffa-1f68-415f-bc28-80e46f37682f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.421240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79fffa-1f68-415f-bc28-80e46f37682f admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.421525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79fffa-1f68-415f-bc28-80e46f37682f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.421797 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79fffa-1f68-415f-bc28-80e46f37682f admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.422201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe0e39-fd34-4f1a-9018-578195b474c6 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.422463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79fffa-1f68-415f-bc28-80e46f37682f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.424265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79fffa-1f68-415f-bc28-80e46f37682f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.428066 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca99419-6a5d-4855-92d9-97873e59825d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.428897 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca99419-6a5d-4855-92d9-97873e59825d admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.429254 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca99419-6a5d-4855-92d9-97873e59825d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.429582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca99419-6a5d-4855-92d9-97873e59825d admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.430421 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca99419-6a5d-4855-92d9-97873e59825d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:06.430863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca99419-6a5d-4855-92d9-97873e59825d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.432767 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca99419-6a5d-4855-92d9-97873e59825d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.437140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1149/2303] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:06.446380 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79fffa-1f68-415f-bc28-80e46f37682f admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.447072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1155/2304] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:19:06.452427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca99419-6a5d-4855-92d9-97873e59825d admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.461496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79fffa-1f68-415f-bc28-80e46f37682f admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.468871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7328012-5239-4518-bde8-352127a9f1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.469522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7328012-5239-4518-bde8-352127a9f1ec admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.469815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7328012-5239-4518-bde8-352127a9f1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.470093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7328012-5239-4518-bde8-352127a9f1ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.470909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7328012-5239-4518-bde8-352127a9f1ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.473425 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7328012-5239-4518-bde8-352127a9f1ec admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.475829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca99419-6a5d-4855-92d9-97873e59825d admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.481715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f1074-b572-414a-9355-f3d53965db83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.482640 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f1074-b572-414a-9355-f3d53965db83 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.483251 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f1074-b572-414a-9355-f3d53965db83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.483780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f1074-b572-414a-9355-f3d53965db83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.484788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f1074-b572-414a-9355-f3d53965db83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.486141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1150/2305] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:06.487872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f1074-b572-414a-9355-f3d53965db83 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.490124 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7328012-5239-4518-bde8-352127a9f1ec admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.501776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7328012-5239-4518-bde8-352127a9f1ec admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.508825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2db31-c3fe-413d-9370-6a6c5499f36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.509490 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2db31-c3fe-413d-9370-6a6c5499f36c admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.509766 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2db31-c3fe-413d-9370-6a6c5499f36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.510099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2db31-c3fe-413d-9370-6a6c5499f36c admin admin] PATH_INFO: `/v3/projects/ee704864f64049158462db9f8ca35504/users/01c288663e6944e88609302a9898f696/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.527070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2db31-c3fe-413d-9370-6a6c5499f36c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee704864f64049158462db9f8ca35504, user_id=01c288663e6944e88609302a9898f696, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.529751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2db31-c3fe-413d-9370-6a6c5499f36c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.531138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1156/2306] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:06 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:06.536640 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f1074-b572-414a-9355-f3d53965db83 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.545633 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1151/2307] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:06 2022] PUT /identity/v3/projects/ee704864f64049158462db9f8ca35504/users/01c288663e6944e88609302a9898f696/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:06.550908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f2469-9fcf-42a9-bc00-191bcc54b18c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.551635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f2469-9fcf-42a9-bc00-191bcc54b18c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.551942 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f2469-9fcf-42a9-bc00-191bcc54b18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.574954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f02f2469-9fcf-42a9-bc00-191bcc54b18c None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:06.576189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f02f2469-9fcf-42a9-bc00-191bcc54b18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:06.613403 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f1074-b572-414a-9355-f3d53965db83 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.649517 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1152/2308] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:06.659341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f2469-9fcf-42a9-bc00-191bcc54b18c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.660843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f02f2469-9fcf-42a9-bc00-191bcc54b18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:06.670540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:06.686524 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14481a-e84d-4905-9257-9553303505a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.687296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14481a-e84d-4905-9257-9553303505a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.687715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14481a-e84d-4905-9257-9553303505a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.688001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14481a-e84d-4905-9257-9553303505a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.688645 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14481a-e84d-4905-9257-9553303505a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.691205 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14481a-e84d-4905-9257-9553303505a6 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.713846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1157/2309] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:06 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:06.717863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14481a-e84d-4905-9257-9553303505a6 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.724417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f2469-9fcf-42a9-bc00-191bcc54b18c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.727938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14481a-e84d-4905-9257-9553303505a6 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.728412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9813-034b-42f1-83c8-73722e3e4d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.729006 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9813-034b-42f1-83c8-73722e3e4d69 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.729278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9813-034b-42f1-83c8-73722e3e4d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.729544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9813-034b-42f1-83c8-73722e3e4d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.733058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423e8ae-6d36-471d-9f2e-9196d3ce382e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.733658 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423e8ae-6d36-471d-9f2e-9196d3ce382e admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.733937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423e8ae-6d36-471d-9f2e-9196d3ce382e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.734214 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423e8ae-6d36-471d-9f2e-9196d3ce382e admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.734859 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423e8ae-6d36-471d-9f2e-9196d3ce382e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.736508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423e8ae-6d36-471d-9f2e-9196d3ce382e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.739838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a9813-034b-42f1-83c8-73722e3e4d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.741288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a9813-034b-42f1-83c8-73722e3e4d69 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.747010 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1158/2310] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:06.750872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423e8ae-6d36-471d-9f2e-9196d3ce382e admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.752213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1153/2311] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:06.761605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423e8ae-6d36-471d-9f2e-9196d3ce382e admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.767119 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6186b5-d9de-40ee-9c78-e1a8a49d8459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.767740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6186b5-d9de-40ee-9c78-e1a8a49d8459 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.768028 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6186b5-d9de-40ee-9c78-e1a8a49d8459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.768368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6186b5-d9de-40ee-9c78-e1a8a49d8459 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.769559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6186b5-d9de-40ee-9c78-e1a8a49d8459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.771218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6186b5-d9de-40ee-9c78-e1a8a49d8459 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.782468 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1159/2312] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:06.786717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9813-034b-42f1-83c8-73722e3e4d69 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:06.796659 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9813-034b-42f1-83c8-73722e3e4d69 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:06.801547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e638b-0faa-4388-90c5-3249655cf538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:06.802202 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e638b-0faa-4388-90c5-3249655cf538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.802486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e638b-0faa-4388-90c5-3249655cf538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.802760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e638b-0faa-4388-90c5-3249655cf538 admin admin] PATH_INFO: `/v3/projects/ecaa74a6ec4c4ad78322fac2212c115c/users/fd823768d62b48c6aabe46d11a6a6233/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.819112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e638b-0faa-4388-90c5-3249655cf538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecaa74a6ec4c4ad78322fac2212c115c, user_id=fd823768d62b48c6aabe46d11a6a6233, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:06.821796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e638b-0faa-4388-90c5-3249655cf538 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:06.838739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1154/2313] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:06 2022] PUT /identity/v3/projects/ecaa74a6ec4c4ad78322fac2212c115c/users/fd823768d62b48c6aabe46d11a6a6233/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:06.849187 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041c8fa-4ab5-4c16-bc20-c0df9a15bfc5 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:06.849514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041c8fa-4ab5-4c16-bc20-c0df9a15bfc5 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:06.849792 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041c8fa-4ab5-4c16-bc20-c0df9a15bfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:06.881444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-4041c8fa-4ab5-4c16-bc20-c0df9a15bfc5 None None] MFA Rules not processed for user `fd823768d62b48c6aabe46d11a6a6233`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:06.882667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4041c8fa-4ab5-4c16-bc20-c0df9a15bfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:06.969533 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1160/2314] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:06.973887 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e638b-0faa-4388-90c5-3249655cf538 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:07.012915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:07.085867 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e638b-0faa-4388-90c5-3249655cf538 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:07.091070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727d437-684d-46cb-ac9b-584c65f65300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:07.091792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727d437-684d-46cb-ac9b-584c65f65300 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.092083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727d437-684d-46cb-ac9b-584c65f65300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.092354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727d437-684d-46cb-ac9b-584c65f65300 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.093011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727d437-684d-46cb-ac9b-584c65f65300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:07.094925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727d437-684d-46cb-ac9b-584c65f65300 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:07.125820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1155/2315] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:06 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:07.130308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041c8fa-4ab5-4c16-bc20-c0df9a15bfc5 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:07.142090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041c8fa-4ab5-4c16-bc20-c0df9a15bfc5 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:07.218079 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb67cab-4834-4a42-acf6-8a9ccf2846b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:07.218933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb67cab-4834-4a42-acf6-8a9ccf2846b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.219311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb67cab-4834-4a42-acf6-8a9ccf2846b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.219658 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb67cab-4834-4a42-acf6-8a9ccf2846b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.220366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb67cab-4834-4a42-acf6-8a9ccf2846b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:07.222422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb67cab-4834-4a42-acf6-8a9ccf2846b0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:07.256550 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1161/2316] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:07 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:07.262147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727d437-684d-46cb-ac9b-584c65f65300 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:07.278580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727d437-684d-46cb-ac9b-584c65f65300 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:07.285231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87323c66-bd21-46f1-a9ec-39c8be520414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:07.286470 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87323c66-bd21-46f1-a9ec-39c8be520414 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.286754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87323c66-bd21-46f1-a9ec-39c8be520414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.287035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87323c66-bd21-46f1-a9ec-39c8be520414 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.287728 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87323c66-bd21-46f1-a9ec-39c8be520414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:07.290315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87323c66-bd21-46f1-a9ec-39c8be520414 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:07.308611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1156/2317] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:07.312451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb67cab-4834-4a42-acf6-8a9ccf2846b0 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:07.323037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb67cab-4834-4a42-acf6-8a9ccf2846b0 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:07.328501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b07fb2-ae60-44a8-9621-b9cc9de10461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:07.329134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b07fb2-ae60-44a8-9621-b9cc9de10461 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.329410 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b07fb2-ae60-44a8-9621-b9cc9de10461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.329698 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b07fb2-ae60-44a8-9621-b9cc9de10461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.330355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b07fb2-ae60-44a8-9621-b9cc9de10461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:07.332065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b07fb2-ae60-44a8-9621-b9cc9de10461 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:07.354315 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1162/2318] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:07.359619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87323c66-bd21-46f1-a9ec-39c8be520414 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:07.379129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87323c66-bd21-46f1-a9ec-39c8be520414 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:07.387846 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342b8e2-768c-448b-84ee-5bb10648c3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:07.389047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342b8e2-768c-448b-84ee-5bb10648c3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.389621 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342b8e2-768c-448b-84ee-5bb10648c3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.390179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342b8e2-768c-448b-84ee-5bb10648c3d8 admin admin] PATH_INFO: `/v3/projects/7800ae001751429485d7c5b2c0bf407d/users/51bb03b2cfa748fdb1a1a4b2ddafd89e/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.418022 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342b8e2-768c-448b-84ee-5bb10648c3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7800ae001751429485d7c5b2c0bf407d, user_id=51bb03b2cfa748fdb1a1a4b2ddafd89e, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:07.423661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342b8e2-768c-448b-84ee-5bb10648c3d8 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:07.442227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1157/2319] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:07 2022] PUT /identity/v3/projects/7800ae001751429485d7c5b2c0bf407d/users/51bb03b2cfa748fdb1a1a4b2ddafd89e/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:07.460026 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be382-6681-4ba6-8ac4-cd449ea45540 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.460855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be382-6681-4ba6-8ac4-cd449ea45540 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.460855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be382-6681-4ba6-8ac4-cd449ea45540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.502376 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a54be382-6681-4ba6-8ac4-cd449ea45540 None None] MFA Rules not processed for user `51bb03b2cfa748fdb1a1a4b2ddafd89e`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:07.504494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a54be382-6681-4ba6-8ac4-cd449ea45540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:07.518272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:07.585253 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1163/2320] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:07.590375 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342b8e2-768c-448b-84ee-5bb10648c3d8 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:07.677395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342b8e2-768c-448b-84ee-5bb10648c3d8 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:07.684299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31005ab2-fcb1-4a3f-84f9-ff30f875f44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:07.685356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31005ab2-fcb1-4a3f-84f9-ff30f875f44e admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.685356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31005ab2-fcb1-4a3f-84f9-ff30f875f44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.685653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31005ab2-fcb1-4a3f-84f9-ff30f875f44e admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.686181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31005ab2-fcb1-4a3f-84f9-ff30f875f44e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:07.688216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31005ab2-fcb1-4a3f-84f9-ff30f875f44e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:07.732223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1158/2321] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:07 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:07.736346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be382-6681-4ba6-8ac4-cd449ea45540 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:07.749600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be382-6681-4ba6-8ac4-cd449ea45540 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:07.793630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c3f86-3c2b-420c-9673-9afef1e87246 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.793935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c3f86-3c2b-420c-9673-9afef1e87246 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.794207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c3f86-3c2b-420c-9673-9afef1e87246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.831898 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-568c3f86-3c2b-420c-9673-9afef1e87246 None None] MFA Rules not processed for user `01c288663e6944e88609302a9898f696`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:07.834463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-568c3f86-3c2b-420c-9673-9afef1e87246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:07.835766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491c8fc-e2e5-4ce4-a1b7-32a7a1141977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:07.836825 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491c8fc-e2e5-4ce4-a1b7-32a7a1141977 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.837288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491c8fc-e2e5-4ce4-a1b7-32a7a1141977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.837891 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491c8fc-e2e5-4ce4-a1b7-32a7a1141977 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.839216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491c8fc-e2e5-4ce4-a1b7-32a7a1141977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:07.844749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491c8fc-e2e5-4ce4-a1b7-32a7a1141977 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:07.897085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1164/2322] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:07 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:07.901810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491c8fc-e2e5-4ce4-a1b7-32a7a1141977 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:07.909683 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:07.914931 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491c8fc-e2e5-4ce4-a1b7-32a7a1141977 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:07.920760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12738b-e3c1-4ab6-83c9-e58d27f6fb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:07.921378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12738b-e3c1-4ab6-83c9-e58d27f6fb5e admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.921662 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12738b-e3c1-4ab6-83c9-e58d27f6fb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.921938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12738b-e3c1-4ab6-83c9-e58d27f6fb5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.922597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12738b-e3c1-4ab6-83c9-e58d27f6fb5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:07.924312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12738b-e3c1-4ab6-83c9-e58d27f6fb5e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:07.930550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1159/2323] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:07.934944 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c3f86-3c2b-420c-9673-9afef1e87246 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:07.938006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1165/2324] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:07.942223 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12738b-e3c1-4ab6-83c9-e58d27f6fb5e admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:07.950809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c3f86-3c2b-420c-9673-9afef1e87246 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:07.954195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12738b-e3c1-4ab6-83c9-e58d27f6fb5e admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:07.956506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871df6b-e626-4929-9340-fa6270a6c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:07.957868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871df6b-e626-4929-9340-fa6270a6c6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.957868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871df6b-e626-4929-9340-fa6270a6c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.957868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871df6b-e626-4929-9340-fa6270a6c6d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.958570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871df6b-e626-4929-9340-fa6270a6c6d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:07.960833 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871df6b-e626-4929-9340-fa6270a6c6d1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:07.962346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cbeea-f1d9-402d-91e5-7769f3323965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:07.963024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cbeea-f1d9-402d-91e5-7769f3323965 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:07.963337 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cbeea-f1d9-402d-91e5-7769f3323965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:07.963616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cbeea-f1d9-402d-91e5-7769f3323965 admin admin] PATH_INFO: `/v3/projects/f75a26c65d384551b81b08bbf870b37e/users/e5cb375716734d8eb2b5aa505a29f545/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:07.985584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cbeea-f1d9-402d-91e5-7769f3323965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f75a26c65d384551b81b08bbf870b37e, user_id=e5cb375716734d8eb2b5aa505a29f545, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:07.994152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cbeea-f1d9-402d-91e5-7769f3323965 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:08.001821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1160/2325] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:07 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:08.006559 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871df6b-e626-4929-9340-fa6270a6c6d1 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:08.015079 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1166/2326] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:07 2022] PUT /identity/v3/projects/f75a26c65d384551b81b08bbf870b37e/users/e5cb375716734d8eb2b5aa505a29f545/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:08.020490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cbeea-f1d9-402d-91e5-7769f3323965 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:08.073606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871df6b-e626-4929-9340-fa6270a6c6d1 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:08.095572 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cbeea-f1d9-402d-91e5-7769f3323965 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:08.100799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626a68c-287f-4896-8c32-89bd2c4e400e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:08.101506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a68c-287f-4896-8c32-89bd2c4e400e admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.101856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a68c-287f-4896-8c32-89bd2c4e400e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.102137 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a68c-287f-4896-8c32-89bd2c4e400e admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.102790 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626a68c-287f-4896-8c32-89bd2c4e400e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:08.104491 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626a68c-287f-4896-8c32-89bd2c4e400e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:08.115399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1167/2327] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:08.119903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626a68c-287f-4896-8c32-89bd2c4e400e admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:08.124986 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:08.133515 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626a68c-287f-4896-8c32-89bd2c4e400e admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:08.135695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f26ed5-969a-4f13-b186-33e73fa7a6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:08.136317 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f26ed5-969a-4f13-b186-33e73fa7a6fd admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.136592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f26ed5-969a-4f13-b186-33e73fa7a6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.136863 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f26ed5-969a-4f13-b186-33e73fa7a6fd admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.137508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f26ed5-969a-4f13-b186-33e73fa7a6fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:08.138842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbfc1c-2cf9-40a3-9d54-5e3daddc9da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:08.139502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f26ed5-969a-4f13-b186-33e73fa7a6fd admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:08.139620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbfc1c-2cf9-40a3-9d54-5e3daddc9da5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.139799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbfc1c-2cf9-40a3-9d54-5e3daddc9da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.140072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbfc1c-2cf9-40a3-9d54-5e3daddc9da5 admin admin] PATH_INFO: `/v3/projects/f75a26c65d384551b81b08bbf870b37e/users/e5cb375716734d8eb2b5aa505a29f545/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.142034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbfc1c-2cf9-40a3-9d54-5e3daddc9da5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f75a26c65d384551b81b08bbf870b37e, user_id=e5cb375716734d8eb2b5aa505a29f545, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:08.145266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbfc1c-2cf9-40a3-9d54-5e3daddc9da5 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:08.156390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1168/2328] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:08 2022] PUT /identity/v3/projects/f75a26c65d384551b81b08bbf870b37e/users/e5cb375716734d8eb2b5aa505a29f545/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:08.168575 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6517ea6-11e1-403b-8343-8e12638c5c7e None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.168909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1161/2329] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:08 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:08.169097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6517ea6-11e1-403b-8343-8e12638c5c7e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.169384 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6517ea6-11e1-403b-8343-8e12638c5c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.172447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f26ed5-969a-4f13-b186-33e73fa7a6fd admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:08.183169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f26ed5-969a-4f13-b186-33e73fa7a6fd admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:08.202135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b6517ea6-11e1-403b-8343-8e12638c5c7e None None] MFA Rules not processed for user `e5cb375716734d8eb2b5aa505a29f545`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:08.203419 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b6517ea6-11e1-403b-8343-8e12638c5c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:08.248587 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc5dde-41b7-43c8-955f-18d8d29234bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:08.249219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc5dde-41b7-43c8-955f-18d8d29234bf admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.249493 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc5dde-41b7-43c8-955f-18d8d29234bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.249765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc5dde-41b7-43c8-955f-18d8d29234bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.250865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc5dde-41b7-43c8-955f-18d8d29234bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:08.253286 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc5dde-41b7-43c8-955f-18d8d29234bf admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:08.266800 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1162/2330] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:08.267904 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc5dde-41b7-43c8-955f-18d8d29234bf admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:08.277790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:08.280639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc5dde-41b7-43c8-955f-18d8d29234bf admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:08.285964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1169/2331] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:08.292196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46822001-66c7-4aab-a9e7-36002518a029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:08.292552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46822001-66c7-4aab-a9e7-36002518a029 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.293169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46822001-66c7-4aab-a9e7-36002518a029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.293892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46822001-66c7-4aab-a9e7-36002518a029 admin admin] PATH_INFO: `/v3/projects/6c696d85df364d04829a7d80b5d7e9aa/users/402624980b5046abb91ca4448ed82417/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.301587 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6517ea6-11e1-403b-8343-8e12638c5c7e None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:08.303922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b6517ea6-11e1-403b-8343-8e12638c5c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:08.314856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46822001-66c7-4aab-a9e7-36002518a029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c696d85df364d04829a7d80b5d7e9aa, user_id=402624980b5046abb91ca4448ed82417, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:08.317793 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46822001-66c7-4aab-a9e7-36002518a029 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:08.332935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1163/2332] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:08 2022] PUT /identity/v3/projects/6c696d85df364d04829a7d80b5d7e9aa/users/402624980b5046abb91ca4448ed82417/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:08.336884 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46822001-66c7-4aab-a9e7-36002518a029 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:08.374575 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6517ea6-11e1-403b-8343-8e12638c5c7e None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:08.404316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46822001-66c7-4aab-a9e7-36002518a029 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:08.408991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d000e36-763f-41b6-869d-95c2f126a51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:08.409620 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d000e36-763f-41b6-869d-95c2f126a51c admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.409876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d000e36-763f-41b6-869d-95c2f126a51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.410145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d000e36-763f-41b6-869d-95c2f126a51c admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.410791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d000e36-763f-41b6-869d-95c2f126a51c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:08.412446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d000e36-763f-41b6-869d-95c2f126a51c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:08.423880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1164/2333] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:08.428391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d000e36-763f-41b6-869d-95c2f126a51c admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:08.438097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d000e36-763f-41b6-869d-95c2f126a51c admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:08.438294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124abd5-43e4-4ea5-857b-d7016bdfae12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:08.438713 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124abd5-43e4-4ea5-857b-d7016bdfae12 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.438998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124abd5-43e4-4ea5-857b-d7016bdfae12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.439414 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124abd5-43e4-4ea5-857b-d7016bdfae12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.443297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28171186-3e8d-4c03-b6a4-8ee1fdc8d475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:08.443941 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28171186-3e8d-4c03-b6a4-8ee1fdc8d475 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:08.444214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28171186-3e8d-4c03-b6a4-8ee1fdc8d475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:08.444487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28171186-3e8d-4c03-b6a4-8ee1fdc8d475 admin admin] PATH_INFO: `/v3/projects/6c696d85df364d04829a7d80b5d7e9aa/users/402624980b5046abb91ca4448ed82417/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:08.446452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28171186-3e8d-4c03-b6a4-8ee1fdc8d475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c696d85df364d04829a7d80b5d7e9aa, user_id=402624980b5046abb91ca4448ed82417, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:08.449756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28171186-3e8d-4c03-b6a4-8ee1fdc8d475 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:08.461093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1165/2334] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:08 2022] PUT /identity/v3/projects/6c696d85df364d04829a7d80b5d7e9aa/users/402624980b5046abb91ca4448ed82417/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:08.486651 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:08.503927 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124abd5-43e4-4ea5-857b-d7016bdfae12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:08.506126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124abd5-43e4-4ea5-857b-d7016bdfae12 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:08.516339 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1170/2335] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:09.306397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643e594-1dbb-429f-b164-831c116c7f00 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:09.306710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643e594-1dbb-429f-b164-831c116c7f00 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:09.307035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643e594-1dbb-429f-b164-831c116c7f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:09.336482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-2643e594-1dbb-429f-b164-831c116c7f00 None None] MFA Rules not processed for user `402624980b5046abb91ca4448ed82417`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:09.337703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2643e594-1dbb-429f-b164-831c116c7f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:09.434577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1166/2336] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:09.445968 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124abd5-43e4-4ea5-857b-d7016bdfae12 service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:09.455389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124abd5-43e4-4ea5-857b-d7016bdfae12 service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:09.539143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f326e-f267-4455-b840-fadf3c07759a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:09.539921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f326e-f267-4455-b840-fadf3c07759a service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:09.540264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f326e-f267-4455-b840-fadf3c07759a service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:09.540568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f326e-f267-4455-b840-fadf3c07759a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:09.616553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f326e-f267-4455-b840-fadf3c07759a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:09.618087 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f326e-f267-4455-b840-fadf3c07759a service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:09.627193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1171/2337] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:10.204831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643e594-1dbb-429f-b164-831c116c7f00 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:10.206805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2643e594-1dbb-429f-b164-831c116c7f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:10.221460 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643e594-1dbb-429f-b164-831c116c7f00 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:10.226072 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ed91b-57a5-413f-829e-cc4800d391bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:10.227092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ed91b-57a5-413f-829e-cc4800d391bd service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:10.227600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ed91b-57a5-413f-829e-cc4800d391bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:10.228222 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ed91b-57a5-413f-829e-cc4800d391bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:10.330353 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:10.347374 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ed91b-57a5-413f-829e-cc4800d391bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:10.348843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ed91b-57a5-413f-829e-cc4800d391bd service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:10.359405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1167/2338] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:11.097293 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f326e-f267-4455-b840-fadf3c07759a service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:11.107842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f326e-f267-4455-b840-fadf3c07759a service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:11.111583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d93528-7aac-4b4e-8f30-6edee0620ed5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:11.112315 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93528-7aac-4b4e-8f30-6edee0620ed5 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:11.112657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93528-7aac-4b4e-8f30-6edee0620ed5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:11.112996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93528-7aac-4b4e-8f30-6edee0620ed5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:11.123310 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d93528-7aac-4b4e-8f30-6edee0620ed5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:11.124750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d93528-7aac-4b4e-8f30-6edee0620ed5 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:11.135929 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1172/2339] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:11.220841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ed91b-57a5-413f-829e-cc4800d391bd service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:11.229183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ed91b-57a5-413f-829e-cc4800d391bd service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:11.233982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afde0c5-edb1-433e-a9bb-80801a10969a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:11.234833 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afde0c5-edb1-433e-a9bb-80801a10969a service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:11.235232 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afde0c5-edb1-433e-a9bb-80801a10969a service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:11.235572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afde0c5-edb1-433e-a9bb-80801a10969a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:11.319463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afde0c5-edb1-433e-a9bb-80801a10969a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:11.320959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afde0c5-edb1-433e-a9bb-80801a10969a service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:11.335230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1168/2340] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:16.674637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d93528-7aac-4b4e-8f30-6edee0620ed5 service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:16.739400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d93528-7aac-4b4e-8f30-6edee0620ed5 service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:16.744040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:16.744669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:16.744948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:16.745219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] PATH_INFO: `/v3/users/fd823768d62b48c6aabe46d11a6a6233` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:16.746323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd823768d62b48c6aabe46d11a6a6233)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:16.748282 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:16.777577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:16.802572 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd823768d62b48c6aabe46d11a6a6233'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.818164 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd823768d62b48c6aabe46d11a6a6233'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.825841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd823768d62b48c6aabe46d11a6a6233'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.826650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd823768d62b48c6aabe46d11a6a6233'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.828139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd823768d62b48c6aabe46d11a6a6233'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:16.843502 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1173/2341] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:16 2022] DELETE /identity/v3/users/fd823768d62b48c6aabe46d11a6a6233 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:16.848683 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fa443-8b3b-4123-a487-12e7757dc953 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:16.849158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fa443-8b3b-4123-a487-12e7757dc953 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:16.849583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fa443-8b3b-4123-a487-12e7757dc953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:16.869867 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-c48fa443-8b3b-4123-a487-12e7757dc953 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:16.871092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c48fa443-8b3b-4123-a487-12e7757dc953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:16.933625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1169/2342] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:16.943604 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:16.945160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:17.028620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ef997-8d13-473a-969f-c1ba7668e57d admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:17.032193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425b02d-1cd9-4275-b38f-90f056308144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.032803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425b02d-1cd9-4275-b38f-90f056308144 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.033083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425b02d-1cd9-4275-b38f-90f056308144 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.033352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425b02d-1cd9-4275-b38f-90f056308144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.043364 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425b02d-1cd9-4275-b38f-90f056308144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.044864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425b02d-1cd9-4275-b38f-90f056308144 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:17.054550 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1174/2343] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:17.572759 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fa443-8b3b-4123-a487-12e7757dc953 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:17.574569 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c48fa443-8b3b-4123-a487-12e7757dc953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:17.588150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fa443-8b3b-4123-a487-12e7757dc953 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:17.593866 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.594539 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.594820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.595400 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] PATH_INFO: `/v3/projects/ecaa74a6ec4c4ad78322fac2212c115c` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.605184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecaa74a6ec4c4ad78322fac2212c115c)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.607364 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:17.671951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecaa74a6ec4c4ad78322fac2212c115c', 'request_id': 'req-7bd14887-cfce-4eac-9d33-d99dd4995a11'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.681080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecaa74a6ec4c4ad78322fac2212c115c', 'request_id': 'req-7bd14887-cfce-4eac-9d33-d99dd4995a11'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.703090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecaa74a6ec4c4ad78322fac2212c115c', 'request_id': 'req-7bd14887-cfce-4eac-9d33-d99dd4995a11'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.707768 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] The token cache is being invalidate because project ecaa74a6ec4c4ad78322fac2212c115c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:17.708062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.708735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.710246 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1170/2344] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:17 2022] DELETE /identity/v3/projects/ecaa74a6ec4c4ad78322fac2212c115c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:17.714680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425b02d-1cd9-4275-b38f-90f056308144 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:17.725768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425b02d-1cd9-4275-b38f-90f056308144 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:17.787254 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:17.787954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:17.788237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:17.788515 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] PATH_INFO: `/v3/users/51bb03b2cfa748fdb1a1a4b2ddafd89e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:17.789668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51bb03b2cfa748fdb1a1a4b2ddafd89e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:17.791649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:17.842995 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51bb03b2cfa748fdb1a1a4b2ddafd89e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.845246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:17.858267 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51bb03b2cfa748fdb1a1a4b2ddafd89e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.866093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51bb03b2cfa748fdb1a1a4b2ddafd89e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.866814 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51bb03b2cfa748fdb1a1a4b2ddafd89e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.867457 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51bb03b2cfa748fdb1a1a4b2ddafd89e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:17.880136 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1175/2345] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:17 2022] DELETE /identity/v3/users/51bb03b2cfa748fdb1a1a4b2ddafd89e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:18.180257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:18.183483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:18.192497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:18.198868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd14887-cfce-4eac-9d33-d99dd4995a11 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:18.257593 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:18.258222 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:18.258498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:18.258767 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] PATH_INFO: `/v3/projects/7800ae001751429485d7c5b2c0bf407d` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:18.259923 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7800ae001751429485d7c5b2c0bf407d)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:18.261825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:18.311429 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7800ae001751429485d7c5b2c0bf407d', 'request_id': 'req-71dc2cf5-9444-4578-91f7-8f3986fcff3c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.318829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7800ae001751429485d7c5b2c0bf407d', 'request_id': 'req-71dc2cf5-9444-4578-91f7-8f3986fcff3c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.324809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7800ae001751429485d7c5b2c0bf407d', 'request_id': 'req-71dc2cf5-9444-4578-91f7-8f3986fcff3c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.336341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] The token cache is being invalidate because project 7800ae001751429485d7c5b2c0bf407d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:18.336632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.337277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.339403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1171/2346] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:18 2022] DELETE /identity/v3/projects/7800ae001751429485d7c5b2c0bf407d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:18.343555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:18.345066 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:18.359181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568b539-9fcc-4981-8baa-26e4740186a0 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:18.416405 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2cc85-46e8-456b-bbf5-17f140fec867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:18.417043 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2cc85-46e8-456b-bbf5-17f140fec867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:18.417321 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2cc85-46e8-456b-bbf5-17f140fec867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:18.417595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2cc85-46e8-456b-bbf5-17f140fec867 admin admin] PATH_INFO: `/v3/users/e5cb375716734d8eb2b5aa505a29f545` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:18.418714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2cc85-46e8-456b-bbf5-17f140fec867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5cb375716734d8eb2b5aa505a29f545)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:18.420680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2cc85-46e8-456b-bbf5-17f140fec867 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:18.466787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f0f2cc85-46e8-456b-bbf5-17f140fec867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5cb375716734d8eb2b5aa505a29f545'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.479420 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f0f2cc85-46e8-456b-bbf5-17f140fec867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5cb375716734d8eb2b5aa505a29f545'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.486577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f0f2cc85-46e8-456b-bbf5-17f140fec867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5cb375716734d8eb2b5aa505a29f545'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.487290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f0f2cc85-46e8-456b-bbf5-17f140fec867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5cb375716734d8eb2b5aa505a29f545'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.487894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f0f2cc85-46e8-456b-bbf5-17f140fec867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5cb375716734d8eb2b5aa505a29f545'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.501136 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1176/2347] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:18 2022] DELETE /identity/v3/users/e5cb375716734d8eb2b5aa505a29f545 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:18.810228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:18.811767 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:18.827671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc2cf5-9444-4578-91f7-8f3986fcff3c admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:18.893878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fff320-0f7e-42a8-b701-311431ee72d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:18.894586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fff320-0f7e-42a8-b701-311431ee72d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:18.894888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fff320-0f7e-42a8-b701-311431ee72d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:18.895199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fff320-0f7e-42a8-b701-311431ee72d1 admin admin] PATH_INFO: `/v3/projects/f75a26c65d384551b81b08bbf870b37e` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:18.896383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fff320-0f7e-42a8-b701-311431ee72d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f75a26c65d384551b81b08bbf870b37e)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:18.898315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fff320-0f7e-42a8-b701-311431ee72d1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:18.937205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:18.948696 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-16fff320-0f7e-42a8-b701-311431ee72d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f75a26c65d384551b81b08bbf870b37e', 'request_id': 'req-16fff320-0f7e-42a8-b701-311431ee72d1'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.956030 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-16fff320-0f7e-42a8-b701-311431ee72d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f75a26c65d384551b81b08bbf870b37e', 'request_id': 'req-16fff320-0f7e-42a8-b701-311431ee72d1'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.962104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-16fff320-0f7e-42a8-b701-311431ee72d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f75a26c65d384551b81b08bbf870b37e', 'request_id': 'req-16fff320-0f7e-42a8-b701-311431ee72d1'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.974890 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-16fff320-0f7e-42a8-b701-311431ee72d1 admin admin] The token cache is being invalidate because project f75a26c65d384551b81b08bbf870b37e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:18.975221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-16fff320-0f7e-42a8-b701-311431ee72d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.975908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-16fff320-0f7e-42a8-b701-311431ee72d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:18.977555 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1172/2348] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:18 2022] DELETE /identity/v3/projects/f75a26c65d384551b81b08bbf870b37e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:18.982934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991e5f2-4270-4d3b-ba59-b850b2995034 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:18.983500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991e5f2-4270-4d3b-ba59-b850b2995034 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:18.983500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991e5f2-4270-4d3b-ba59-b850b2995034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.004309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-5991e5f2-4270-4d3b-ba59-b850b2995034 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:19.005534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5991e5f2-4270-4d3b-ba59-b850b2995034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:19.048367 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:19.068990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1177/2349] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:19.076937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3904414-064f-48e1-acf1-d2acd13f70da None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.077215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3904414-064f-48e1-acf1-d2acd13f70da None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.077486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3904414-064f-48e1-acf1-d2acd13f70da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.103821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-e3904414-064f-48e1-acf1-d2acd13f70da None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:19.105318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e3904414-064f-48e1-acf1-d2acd13f70da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:19.117053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1173/2350] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:19.122354 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991e5f2-4270-4d3b-ba59-b850b2995034 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:19.133338 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991e5f2-4270-4d3b-ba59-b850b2995034 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.138685 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5f9dd-dcc7-4b83-9a17-6dfe86df735f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.139586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5f9dd-dcc7-4b83-9a17-6dfe86df735f admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.139905 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5f9dd-dcc7-4b83-9a17-6dfe86df735f admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.140188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5f9dd-dcc7-4b83-9a17-6dfe86df735f admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.140878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5f9dd-dcc7-4b83-9a17-6dfe86df735f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:19.141186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5f9dd-dcc7-4b83-9a17-6dfe86df735f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.142797 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5f9dd-dcc7-4b83-9a17-6dfe86df735f admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.155378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1178/2351] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:19.159776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3904414-064f-48e1-acf1-d2acd13f70da None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:19.169056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3904414-064f-48e1-acf1-d2acd13f70da None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.173717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e5be3-3806-42ad-8396-cfe9fe38c348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.174349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e5be3-3806-42ad-8396-cfe9fe38c348 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.174628 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e5be3-3806-42ad-8396-cfe9fe38c348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.175471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e5be3-3806-42ad-8396-cfe9fe38c348 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.176112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e5be3-3806-42ad-8396-cfe9fe38c348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.178012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e5be3-3806-42ad-8396-cfe9fe38c348 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.211214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1174/2352] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:19 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:19.213092 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5f9dd-dcc7-4b83-9a17-6dfe86df735f admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:19.224060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5f9dd-dcc7-4b83-9a17-6dfe86df735f admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.287844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816c8d3-e491-4445-8774-003c3e90a1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.288481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816c8d3-e491-4445-8774-003c3e90a1ea admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.288760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816c8d3-e491-4445-8774-003c3e90a1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.289031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816c8d3-e491-4445-8774-003c3e90a1ea admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.289672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816c8d3-e491-4445-8774-003c3e90a1ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.291681 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816c8d3-e491-4445-8774-003c3e90a1ea admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.318504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1179/2353] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:19 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:19:19.320001 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e5be3-3806-42ad-8396-cfe9fe38c348 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:19.330509 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e5be3-3806-42ad-8396-cfe9fe38c348 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.335681 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949e420-8009-4cdc-95a2-c536882a4fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.336311 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949e420-8009-4cdc-95a2-c536882a4fec admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.336589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949e420-8009-4cdc-95a2-c536882a4fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.336997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949e420-8009-4cdc-95a2-c536882a4fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.337657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949e420-8009-4cdc-95a2-c536882a4fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.339417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949e420-8009-4cdc-95a2-c536882a4fec admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.363671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1175/2354] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:19.363801 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816c8d3-e491-4445-8774-003c3e90a1ea admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:19.366213 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816c8d3-e491-4445-8774-003c3e90a1ea admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.371469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296988e-887f-4be0-ae8d-5c054646f62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.372108 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296988e-887f-4be0-ae8d-5c054646f62e admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.372390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296988e-887f-4be0-ae8d-5c054646f62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.372667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296988e-887f-4be0-ae8d-5c054646f62e admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.373317 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296988e-887f-4be0-ae8d-5c054646f62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.374904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296988e-887f-4be0-ae8d-5c054646f62e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.386939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1180/2355] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:19.391178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949e420-8009-4cdc-95a2-c536882a4fec admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:19.401450 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949e420-8009-4cdc-95a2-c536882a4fec admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.406284 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f547c5-fd25-44dc-b02b-31c285b99cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.406942 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f547c5-fd25-44dc-b02b-31c285b99cca admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.407424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f547c5-fd25-44dc-b02b-31c285b99cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.407711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f547c5-fd25-44dc-b02b-31c285b99cca admin admin] PATH_INFO: `/v3/projects/963b95435fe545bb9e9ff7a1c4490a05/users/57896ea1ff2046d282fe47e2a8191676/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.424019 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f547c5-fd25-44dc-b02b-31c285b99cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=963b95435fe545bb9e9ff7a1c4490a05, user_id=57896ea1ff2046d282fe47e2a8191676, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.427334 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f547c5-fd25-44dc-b02b-31c285b99cca admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.441623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1176/2356] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:19 2022] PUT /identity/v3/projects/963b95435fe545bb9e9ff7a1c4490a05/users/57896ea1ff2046d282fe47e2a8191676/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:19.446985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93396a8e-9311-4ea9-890f-0132b671ae2c None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.447367 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93396a8e-9311-4ea9-890f-0132b671ae2c None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.447713 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93396a8e-9311-4ea9-890f-0132b671ae2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.470131 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-93396a8e-9311-4ea9-890f-0132b671ae2c None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:19.471486 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-93396a8e-9311-4ea9-890f-0132b671ae2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:19.533678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1181/2357] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:19.544124 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f547c5-fd25-44dc-b02b-31c285b99cca admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:19.545891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f9f547c5-fd25-44dc-b02b-31c285b99cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:19.615261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f547c5-fd25-44dc-b02b-31c285b99cca admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:19.619214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b90ed-d916-4286-827c-39fe0bc33dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:19.619826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b90ed-d916-4286-827c-39fe0bc33dff service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:19.620103 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b90ed-d916-4286-827c-39fe0bc33dff service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:19.620374 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b90ed-d916-4286-827c-39fe0bc33dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:19.630413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b90ed-d916-4286-827c-39fe0bc33dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:19.631984 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b90ed-d916-4286-827c-39fe0bc33dff service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:19.642834 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1177/2358] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:23.162620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeffa1c-249b-4b39-9fe7-88dda8043559 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:23.163492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeffa1c-249b-4b39-9fe7-88dda8043559 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:23.164316 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeffa1c-249b-4b39-9fe7-88dda8043559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:23.192888 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-5aeffa1c-249b-4b39-9fe7-88dda8043559 None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:23.195574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5aeffa1c-249b-4b39-9fe7-88dda8043559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:23.306766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:23.323981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1182/2359] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:19:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:23.436088 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b90ed-d916-4286-827c-39fe0bc33dff service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:23.438078 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-915b90ed-d916-4286-827c-39fe0bc33dff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:23.626910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:23.647157 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b90ed-d916-4286-827c-39fe0bc33dff service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:23.656794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41863e02-4010-42bd-b8e3-ab7983d0eb79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:23.657443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41863e02-4010-42bd-b8e3-ab7983d0eb79 service swift] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:23.657724 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41863e02-4010-42bd-b8e3-ab7983d0eb79 service swift] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:23.657997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41863e02-4010-42bd-b8e3-ab7983d0eb79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:23.672906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41863e02-4010-42bd-b8e3-ab7983d0eb79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:23.675950 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41863e02-4010-42bd-b8e3-ab7983d0eb79 service swift] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:23.694097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1178/2360] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:19:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:19:24.286227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441137f3-7fef-40ef-b490-59b1dbb0d2d0 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:24.287157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441137f3-7fef-40ef-b490-59b1dbb0d2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:24.287432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441137f3-7fef-40ef-b490-59b1dbb0d2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:24.317353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-441137f3-7fef-40ef-b490-59b1dbb0d2d0 None None] MFA Rules not processed for user `57896ea1ff2046d282fe47e2a8191676`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:24.318603 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-441137f3-7fef-40ef-b490-59b1dbb0d2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:24.402378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1183/2361] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:24.406762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41863e02-4010-42bd-b8e3-ab7983d0eb79 service swift] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:24.419192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41863e02-4010-42bd-b8e3-ab7983d0eb79 service swift] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:24.424698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614c5d6-3477-4fe2-9443-d8fed5552eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:24.425323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614c5d6-3477-4fe2-9443-d8fed5552eed admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:24.425795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614c5d6-3477-4fe2-9443-d8fed5552eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:24.426232 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614c5d6-3477-4fe2-9443-d8fed5552eed admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:24.427276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614c5d6-3477-4fe2-9443-d8fed5552eed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:24.429223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614c5d6-3477-4fe2-9443-d8fed5552eed admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:24.459724 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1179/2362] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:24 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:24.463886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441137f3-7fef-40ef-b490-59b1dbb0d2d0 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:24.477316 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441137f3-7fef-40ef-b490-59b1dbb0d2d0 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:24.549360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9b3e7-bff4-4f53-8827-cff3282b1d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:24.550085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9b3e7-bff4-4f53-8827-cff3282b1d72 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:24.550488 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9b3e7-bff4-4f53-8827-cff3282b1d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:24.550860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9b3e7-bff4-4f53-8827-cff3282b1d72 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:24.551727 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9b3e7-bff4-4f53-8827-cff3282b1d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:24.553881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9b3e7-bff4-4f53-8827-cff3282b1d72 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:24.586549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1184/2363] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:24 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:24.587899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614c5d6-3477-4fe2-9443-d8fed5552eed admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:24.600086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614c5d6-3477-4fe2-9443-d8fed5552eed admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:24.606394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecb0c1-f4b9-40b2-9723-c906d13e12e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:24.606815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecb0c1-f4b9-40b2-9723-c906d13e12e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:24.607098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecb0c1-f4b9-40b2-9723-c906d13e12e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:24.607401 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecb0c1-f4b9-40b2-9723-c906d13e12e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:24.608065 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecb0c1-f4b9-40b2-9723-c906d13e12e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:24.609699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ecb0c1-f4b9-40b2-9723-c906d13e12e1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:24.622043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1180/2364] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:24.626324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9b3e7-bff4-4f53-8827-cff3282b1d72 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:24.637527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9b3e7-bff4-4f53-8827-cff3282b1d72 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:24.642262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39e924-ba74-46d1-ba62-e71e53ee68f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:24.642929 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39e924-ba74-46d1-ba62-e71e53ee68f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:24.644079 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39e924-ba74-46d1-ba62-e71e53ee68f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:24.644380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39e924-ba74-46d1-ba62-e71e53ee68f9 admin admin] PATH_INFO: `/v3/projects/1c95a078e8624467b76a9d3aef768a7e/users/9b571dfc031e46e8b003859ccaa74641/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:24.663843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39e924-ba74-46d1-ba62-e71e53ee68f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c95a078e8624467b76a9d3aef768a7e, user_id=9b571dfc031e46e8b003859ccaa74641, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:24.667349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39e924-ba74-46d1-ba62-e71e53ee68f9 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:24.686755 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1185/2365] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:24 2022] PUT /identity/v3/projects/1c95a078e8624467b76a9d3aef768a7e/users/9b571dfc031e46e8b003859ccaa74641/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:24.692781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecb0c1-f4b9-40b2-9723-c906d13e12e1 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:24.711131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecb0c1-f4b9-40b2-9723-c906d13e12e1 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:24.817866 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712876f-304e-4745-822e-347f015ee48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:24.818660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712876f-304e-4745-822e-347f015ee48c admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:24.818949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712876f-304e-4745-822e-347f015ee48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:24.820003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712876f-304e-4745-822e-347f015ee48c admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:24.820003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712876f-304e-4745-822e-347f015ee48c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:24.821605 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712876f-304e-4745-822e-347f015ee48c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:24.834996 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1181/2366] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:24.839407 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39e924-ba74-46d1-ba62-e71e53ee68f9 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:24.851215 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39e924-ba74-46d1-ba62-e71e53ee68f9 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:24.860854 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b97e0-2fe2-4d45-b940-57a2f73dde41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:24.861543 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b97e0-2fe2-4d45-b940-57a2f73dde41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:24.862371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b97e0-2fe2-4d45-b940-57a2f73dde41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:24.862671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b97e0-2fe2-4d45-b940-57a2f73dde41 admin admin] PATH_INFO: `/v3/projects/1c95a078e8624467b76a9d3aef768a7e/users/9b571dfc031e46e8b003859ccaa74641/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:24.867204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b97e0-2fe2-4d45-b940-57a2f73dde41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c95a078e8624467b76a9d3aef768a7e, user_id=9b571dfc031e46e8b003859ccaa74641, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:24.870900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b97e0-2fe2-4d45-b940-57a2f73dde41 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:24.891207 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1186/2367] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:24 2022] PUT /identity/v3/projects/1c95a078e8624467b76a9d3aef768a7e/users/9b571dfc031e46e8b003859ccaa74641/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:32.749484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fb3cb-d62d-4134-a274-3a7cd1d24815 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:32.750491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fb3cb-d62d-4134-a274-3a7cd1d24815 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:32.751256 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fb3cb-d62d-4134-a274-3a7cd1d24815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:32.798506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ca0fb3cb-d62d-4134-a274-3a7cd1d24815 None None] MFA Rules not processed for user `9b571dfc031e46e8b003859ccaa74641`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:32.805711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ca0fb3cb-d62d-4134-a274-3a7cd1d24815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:32.998267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1182/2368] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:33.042554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b97e0-2fe2-4d45-b940-57a2f73dde41 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:33.045375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2d0b97e0-2fe2-4d45-b940-57a2f73dde41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:33.232107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:33.365728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b97e0-2fe2-4d45-b940-57a2f73dde41 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:33.382022 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819d57b-06d5-4ebc-ae05-248d5c346796 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:33.382749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819d57b-06d5-4ebc-ae05-248d5c346796 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:33.383431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819d57b-06d5-4ebc-ae05-248d5c346796 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:33.383431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819d57b-06d5-4ebc-ae05-248d5c346796 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:33.540679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fb3cb-d62d-4134-a274-3a7cd1d24815 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:33.542325 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819d57b-06d5-4ebc-ae05-248d5c346796 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:33.544225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ca0fb3cb-d62d-4134-a274-3a7cd1d24815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:33.547159 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819d57b-06d5-4ebc-ae05-248d5c346796 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:33.561926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1187/2369] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:33.655573 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:33.713962 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fb3cb-d62d-4134-a274-3a7cd1d24815 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:33.719871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:33.723633 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:33.725493 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:33.726025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] PATH_INFO: `/v3/users/01c288663e6944e88609302a9898f696` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:33.732413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c288663e6944e88609302a9898f696)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:33.735796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:33.807424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c288663e6944e88609302a9898f696'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:33.816339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c288663e6944e88609302a9898f696'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:33.816945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c288663e6944e88609302a9898f696'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:33.817574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c288663e6944e88609302a9898f696'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:33.831636 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c288663e6944e88609302a9898f696'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:33.850349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1183/2370] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:33 2022] DELETE /identity/v3/users/01c288663e6944e88609302a9898f696 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:34.094268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819d57b-06d5-4ebc-ae05-248d5c346796 service nova] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:34.108210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819d57b-06d5-4ebc-ae05-248d5c346796 service nova] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:34.210625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:34.211247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:34.211533 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:34.211810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] PATH_INFO: `/v3/projects/ee704864f64049158462db9f8ca35504` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:34.212986 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee704864f64049158462db9f8ca35504)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:34.214951 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:34.295483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee704864f64049158462db9f8ca35504', 'request_id': 'req-51a07319-55b5-4e35-bb4a-8d37845fbaf4'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:34.304721 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee704864f64049158462db9f8ca35504', 'request_id': 'req-51a07319-55b5-4e35-bb4a-8d37845fbaf4'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:34.319225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee704864f64049158462db9f8ca35504', 'request_id': 'req-51a07319-55b5-4e35-bb4a-8d37845fbaf4'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:34.330992 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] The token cache is being invalidate because project ee704864f64049158462db9f8ca35504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:34.331418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:34.332113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:34.334078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1188/2371] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:34 2022] DELETE /identity/v3/projects/ee704864f64049158462db9f8ca35504 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:34.339578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:34.341132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:34.360205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c1edf-d9db-4212-881f-69e89e2a7c9f admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:34.457114 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6ad0b-9d21-43ba-b944-ed814f7a9920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:34.457732 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6ad0b-9d21-43ba-b944-ed814f7a9920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:34.458034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6ad0b-9d21-43ba-b944-ed814f7a9920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:34.458303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6ad0b-9d21-43ba-b944-ed814f7a9920 admin admin] PATH_INFO: `/v3/users/402624980b5046abb91ca4448ed82417` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:34.464676 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6ad0b-9d21-43ba-b944-ed814f7a9920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=402624980b5046abb91ca4448ed82417)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:34.466738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6ad0b-9d21-43ba-b944-ed814f7a9920 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:34.570780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fae6ad0b-9d21-43ba-b944-ed814f7a9920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '402624980b5046abb91ca4448ed82417'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:34.607609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fae6ad0b-9d21-43ba-b944-ed814f7a9920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '402624980b5046abb91ca4448ed82417'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:34.608357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fae6ad0b-9d21-43ba-b944-ed814f7a9920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '402624980b5046abb91ca4448ed82417'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:34.609096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fae6ad0b-9d21-43ba-b944-ed814f7a9920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '402624980b5046abb91ca4448ed82417'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:34.636445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-fae6ad0b-9d21-43ba-b944-ed814f7a9920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '402624980b5046abb91ca4448ed82417'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:34.664765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:34.701238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1184/2372] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:34 2022] DELETE /identity/v3/users/402624980b5046abb91ca4448ed82417 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:35.134642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:35.136789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:35.163530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a07319-55b5-4e35-bb4a-8d37845fbaf4 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:35.175529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:35.297118 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027984b7-e981-4690-8c03-7dc7cf8fc3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:35.298380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027984b7-e981-4690-8c03-7dc7cf8fc3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:35.299632 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027984b7-e981-4690-8c03-7dc7cf8fc3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:35.300475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027984b7-e981-4690-8c03-7dc7cf8fc3ef admin admin] PATH_INFO: `/v3/projects/6c696d85df364d04829a7d80b5d7e9aa` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:35.302430 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027984b7-e981-4690-8c03-7dc7cf8fc3ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c696d85df364d04829a7d80b5d7e9aa)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:35.304765 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027984b7-e981-4690-8c03-7dc7cf8fc3ef admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:35.400570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-027984b7-e981-4690-8c03-7dc7cf8fc3ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c696d85df364d04829a7d80b5d7e9aa', 'request_id': 'req-027984b7-e981-4690-8c03-7dc7cf8fc3ef'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:35.417618 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-027984b7-e981-4690-8c03-7dc7cf8fc3ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c696d85df364d04829a7d80b5d7e9aa', 'request_id': 'req-027984b7-e981-4690-8c03-7dc7cf8fc3ef'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:35.437061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-027984b7-e981-4690-8c03-7dc7cf8fc3ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c696d85df364d04829a7d80b5d7e9aa', 'request_id': 'req-027984b7-e981-4690-8c03-7dc7cf8fc3ef'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:35.468344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-027984b7-e981-4690-8c03-7dc7cf8fc3ef admin admin] The token cache is being invalidate because project 6c696d85df364d04829a7d80b5d7e9aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:35.470167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-027984b7-e981-4690-8c03-7dc7cf8fc3ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:35.471773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-027984b7-e981-4690-8c03-7dc7cf8fc3ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:35.474973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1189/2373] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:35 2022] DELETE /identity/v3/projects/6c696d85df364d04829a7d80b5d7e9aa => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:35.481631 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a9ef9-7655-416e-bea5-5149e249b13b None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:35.481631 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a9ef9-7655-416e-bea5-5149e249b13b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:35.481631 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a9ef9-7655-416e-bea5-5149e249b13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:35.527534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f38a9ef9-7655-416e-bea5-5149e249b13b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:35.528799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f38a9ef9-7655-416e-bea5-5149e249b13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:35.667761 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1185/2374] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:35.678250 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc75ee-13c9-4c20-b2f9-499f536ad839 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:35.678463 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc75ee-13c9-4c20-b2f9-499f536ad839 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:35.678885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc75ee-13c9-4c20-b2f9-499f536ad839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:35.727801 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-fbfc75ee-13c9-4c20-b2f9-499f536ad839 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:35.729342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-fbfc75ee-13c9-4c20-b2f9-499f536ad839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:35.759380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1190/2375] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:35.771136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a9ef9-7655-416e-bea5-5149e249b13b None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:35.785341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a9ef9-7655-416e-bea5-5149e249b13b None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:35.805691 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e20fde-973c-4c23-acf1-e1830ff67fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:35.805938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e20fde-973c-4c23-acf1-e1830ff67fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:35.806160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e20fde-973c-4c23-acf1-e1830ff67fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:35.806609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e20fde-973c-4c23-acf1-e1830ff67fe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:35.807082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e20fde-973c-4c23-acf1-e1830ff67fe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:35.807441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e20fde-973c-4c23-acf1-e1830ff67fe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:35.810480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e20fde-973c-4c23-acf1-e1830ff67fe1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:35.849201 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1186/2376] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:35.850709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc75ee-13c9-4c20-b2f9-499f536ad839 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:35.891922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc75ee-13c9-4c20-b2f9-499f536ad839 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:35.898389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c17897-d8d5-44ae-8617-69d1c2952feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:35.899203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c17897-d8d5-44ae-8617-69d1c2952feb admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:35.899534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c17897-d8d5-44ae-8617-69d1c2952feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:35.899883 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c17897-d8d5-44ae-8617-69d1c2952feb admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:35.900705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c17897-d8d5-44ae-8617-69d1c2952feb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:35.902771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c17897-d8d5-44ae-8617-69d1c2952feb admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:35.951161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1191/2377] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:35 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:35.955942 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e20fde-973c-4c23-acf1-e1830ff67fe1 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:35.971423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e20fde-973c-4c23-acf1-e1830ff67fe1 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:36.095879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87019ee3-827f-4e04-ada8-c1d4e2e12c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:36.096567 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87019ee3-827f-4e04-ada8-c1d4e2e12c42 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:36.096895 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87019ee3-827f-4e04-ada8-c1d4e2e12c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:36.097239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87019ee3-827f-4e04-ada8-c1d4e2e12c42 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:36.098017 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87019ee3-827f-4e04-ada8-c1d4e2e12c42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:36.100364 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87019ee3-827f-4e04-ada8-c1d4e2e12c42 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:36.133127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1187/2378] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:35 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:36.137134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c17897-d8d5-44ae-8617-69d1c2952feb admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:36.154570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c17897-d8d5-44ae-8617-69d1c2952feb admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:36.163510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a11f75-132f-4f2d-a226-1f2749eb8383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:36.164150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11f75-132f-4f2d-a226-1f2749eb8383 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:36.164624 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11f75-132f-4f2d-a226-1f2749eb8383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:36.165072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11f75-132f-4f2d-a226-1f2749eb8383 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:36.165926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a11f75-132f-4f2d-a226-1f2749eb8383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:36.167846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a11f75-132f-4f2d-a226-1f2749eb8383 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:36.184066 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1192/2379] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:36.187053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87019ee3-827f-4e04-ada8-c1d4e2e12c42 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:36.198811 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87019ee3-827f-4e04-ada8-c1d4e2e12c42 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:36.204471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e719ec7-baf8-4314-8260-67755b5a9764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:36.205136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e719ec7-baf8-4314-8260-67755b5a9764 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:36.205421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e719ec7-baf8-4314-8260-67755b5a9764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:36.205701 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e719ec7-baf8-4314-8260-67755b5a9764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:36.206355 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e719ec7-baf8-4314-8260-67755b5a9764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:36.208056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e719ec7-baf8-4314-8260-67755b5a9764 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:36.220897 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1188/2380] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:19:36.225304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a11f75-132f-4f2d-a226-1f2749eb8383 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:36.237013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a11f75-132f-4f2d-a226-1f2749eb8383 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:36.242095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d01de-5a11-4e6a-a021-7ac3e03d66ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:36.243053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d01de-5a11-4e6a-a021-7ac3e03d66ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:36.243455 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d01de-5a11-4e6a-a021-7ac3e03d66ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:36.243733 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d01de-5a11-4e6a-a021-7ac3e03d66ad admin admin] PATH_INFO: `/v3/projects/1d6cde514fcb479a943c665edd186b61/users/7992effda92b4b1592b0687099c485b7/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:36.265007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d01de-5a11-4e6a-a021-7ac3e03d66ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d6cde514fcb479a943c665edd186b61, user_id=7992effda92b4b1592b0687099c485b7, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:36.268086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d01de-5a11-4e6a-a021-7ac3e03d66ad admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:36.285849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1193/2381] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:36 2022] PUT /identity/v3/projects/1d6cde514fcb479a943c665edd186b61/users/7992effda92b4b1592b0687099c485b7/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:19:36.298131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8cf41-562a-4046-aa1a-cc405c07a30a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:36.298423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8cf41-562a-4046-aa1a-cc405c07a30a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:36.299518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8cf41-562a-4046-aa1a-cc405c07a30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:36.356955 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-17a8cf41-562a-4046-aa1a-cc405c07a30a None None] MFA Rules not processed for user `7992effda92b4b1592b0687099c485b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:36.364688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-17a8cf41-562a-4046-aa1a-cc405c07a30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:36.414771 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:36.455059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1189/2382] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:36.469902 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d01de-5a11-4e6a-a021-7ac3e03d66ad admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:36.477323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d99d01de-5a11-4e6a-a021-7ac3e03d66ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:36.523551 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:36.560195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d01de-5a11-4e6a-a021-7ac3e03d66ad admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:36.564869 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f993d78-22bc-48df-a153-6e167cdcae79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:36.565651 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f993d78-22bc-48df-a153-6e167cdcae79 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:36.565936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f993d78-22bc-48df-a153-6e167cdcae79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:36.566397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f993d78-22bc-48df-a153-6e167cdcae79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:36.578926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f993d78-22bc-48df-a153-6e167cdcae79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:36.580537 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f993d78-22bc-48df-a153-6e167cdcae79 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:36.591626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1194/2383] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:36.613752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8cf41-562a-4046-aa1a-cc405c07a30a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:36.725037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8cf41-562a-4046-aa1a-cc405c07a30a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:36.733838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:36.734648 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:36.735026 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:36.735431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] PATH_INFO: `/v3/users/7992effda92b4b1592b0687099c485b7` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:36.737966 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7992effda92b4b1592b0687099c485b7)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:36.741589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:36.843710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7992effda92b4b1592b0687099c485b7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:36.865990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7992effda92b4b1592b0687099c485b7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:36.867142 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7992effda92b4b1592b0687099c485b7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:36.868102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7992effda92b4b1592b0687099c485b7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:36.887411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7992effda92b4b1592b0687099c485b7'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:36.925390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1190/2384] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:19:36 2022] DELETE /identity/v3/users/7992effda92b4b1592b0687099c485b7 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:36.930302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead4f78-82e4-44dd-ae48-fde6bf46a677 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:36.930673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead4f78-82e4-44dd-ae48-fde6bf46a677 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:36.931005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead4f78-82e4-44dd-ae48-fde6bf46a677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:36.963854 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-4ead4f78-82e4-44dd-ae48-fde6bf46a677 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:36.965606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4ead4f78-82e4-44dd-ae48-fde6bf46a677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:37.069592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1195/2385] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:37.090330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:37.093505 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:37.179851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc2658-86a4-42ad-b72a-f50229e48f97 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:37.186091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b74a6-2602-40f0-9212-c993d1c06f5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:37.186879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b74a6-2602-40f0-9212-c993d1c06f5b service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:37.187380 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b74a6-2602-40f0-9212-c993d1c06f5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:37.187512 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b74a6-2602-40f0-9212-c993d1c06f5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:37.207334 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:37.215704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b74a6-2602-40f0-9212-c993d1c06f5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:37.218533 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b74a6-2602-40f0-9212-c993d1c06f5b service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:37.248814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1191/2386] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:19:37.752322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead4f78-82e4-44dd-ae48-fde6bf46a677 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:37.754891 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4ead4f78-82e4-44dd-ae48-fde6bf46a677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:37.796156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead4f78-82e4-44dd-ae48-fde6bf46a677 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:37.802772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:37.803486 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:37.804042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:37.804306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3 admin admin] PATH_INFO: `/v3/projects/1d6cde514fcb479a943c665edd186b61` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:37.807148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d6cde514fcb479a943c665edd186b61)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:37.809316 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:37.882351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d6cde514fcb479a943c665edd186b61', 'request_id': 'req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:37.893078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d6cde514fcb479a943c665edd186b61', 'request_id': 'req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:37.904169 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d6cde514fcb479a943c665edd186b61', 'request_id': 'req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:37.931526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3 admin admin] The token cache is being invalidate because project 1d6cde514fcb479a943c665edd186b61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:19:37.931978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:37.932974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-269a9c6d-1a3f-4a06-ab7b-ae07378454d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:19:37.935283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1196/2387] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:19:37 2022] DELETE /identity/v3/projects/1d6cde514fcb479a943c665edd186b61 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:37.959739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bba7ef-1812-4da1-9208-bef6c78698ff None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:37.960448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bba7ef-1812-4da1-9208-bef6c78698ff None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:37.960448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bba7ef-1812-4da1-9208-bef6c78698ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:38.020128 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f0bba7ef-1812-4da1-9208-bef6c78698ff None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:38.021545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f0bba7ef-1812-4da1-9208-bef6c78698ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:38.172314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1192/2388] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:38.181344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd4888-c2a0-4bdf-b889-1bc6a44e7171 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:38.181764 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd4888-c2a0-4bdf-b889-1bc6a44e7171 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:38.182193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd4888-c2a0-4bdf-b889-1bc6a44e7171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:38.242559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-08cd4888-c2a0-4bdf-b889-1bc6a44e7171 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:38.244335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-08cd4888-c2a0-4bdf-b889-1bc6a44e7171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:38.259856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1197/2389] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:19:38.266164 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bba7ef-1812-4da1-9208-bef6c78698ff None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:38.281736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bba7ef-1812-4da1-9208-bef6c78698ff None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:38.298361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657e542-3ac2-4615-b780-920ba7e6ed6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:38.299752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657e542-3ac2-4615-b780-920ba7e6ed6a admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:38.300184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657e542-3ac2-4615-b780-920ba7e6ed6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:38.300605 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657e542-3ac2-4615-b780-920ba7e6ed6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:38.301827 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657e542-3ac2-4615-b780-920ba7e6ed6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:19:38.302280 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657e542-3ac2-4615-b780-920ba7e6ed6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:38.304237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657e542-3ac2-4615-b780-920ba7e6ed6a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:38.323163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1193/2390] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:19:38.327762 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd4888-c2a0-4bdf-b889-1bc6a44e7171 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:38.350708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd4888-c2a0-4bdf-b889-1bc6a44e7171 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:38.358507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8670d99-680d-4f14-a66c-2ec92e0d22b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:38.359985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8670d99-680d-4f14-a66c-2ec92e0d22b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:38.360493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8670d99-680d-4f14-a66c-2ec92e0d22b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:38.361861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8670d99-680d-4f14-a66c-2ec92e0d22b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:38.363341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8670d99-680d-4f14-a66c-2ec92e0d22b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:38.365992 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8670d99-680d-4f14-a66c-2ec92e0d22b0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:38.437192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657e542-3ac2-4615-b780-920ba7e6ed6a admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:38.450967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1198/2391] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:19:38 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:38.465861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657e542-3ac2-4615-b780-920ba7e6ed6a admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:38.556877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5876361-a232-4d17-b591-a3f93c8f079d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:38.557551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5876361-a232-4d17-b591-a3f93c8f079d admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:38.557887 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5876361-a232-4d17-b591-a3f93c8f079d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:38.558320 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5876361-a232-4d17-b591-a3f93c8f079d admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:38.559419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5876361-a232-4d17-b591-a3f93c8f079d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:38.563915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5876361-a232-4d17-b591-a3f93c8f079d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:38.607150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1194/2392] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:19:38 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:19:38.610780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8670d99-680d-4f14-a66c-2ec92e0d22b0 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:38.623255 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8670d99-680d-4f14-a66c-2ec92e0d22b0 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:38.629249 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959219a3-91a7-4825-889b-b0643f83c210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:38.629874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959219a3-91a7-4825-889b-b0643f83c210 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:38.630226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959219a3-91a7-4825-889b-b0643f83c210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:38.630492 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959219a3-91a7-4825-889b-b0643f83c210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:38.633652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959219a3-91a7-4825-889b-b0643f83c210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:38.635977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959219a3-91a7-4825-889b-b0643f83c210 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:38.655477 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1199/2393] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:38.657096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5876361-a232-4d17-b591-a3f93c8f079d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:38.676933 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5876361-a232-4d17-b591-a3f93c8f079d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:38.684987 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769745ea-e12f-4038-bb4f-d2601e970c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:38.685779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769745ea-e12f-4038-bb4f-d2601e970c3a admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:38.686350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769745ea-e12f-4038-bb4f-d2601e970c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:38.686350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769745ea-e12f-4038-bb4f-d2601e970c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:38.686988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769745ea-e12f-4038-bb4f-d2601e970c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:38.688893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769745ea-e12f-4038-bb4f-d2601e970c3a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:38.721885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1195/2394] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:19:38.725920 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959219a3-91a7-4825-889b-b0643f83c210 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:38.740027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959219a3-91a7-4825-889b-b0643f83c210 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:38.745793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1ed5d-8146-41ed-9f18-a8f3dc59b692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:38.746545 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1ed5d-8146-41ed-9f18-a8f3dc59b692 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:38.746947 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1ed5d-8146-41ed-9f18-a8f3dc59b692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:38.747372 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1ed5d-8146-41ed-9f18-a8f3dc59b692 admin admin] PATH_INFO: `/v3/projects/476c69341d6b46499a50461862fd8a04/users/f1ed594b5cb9438e9328335261e9997f/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:38.766107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1ed5d-8146-41ed-9f18-a8f3dc59b692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=476c69341d6b46499a50461862fd8a04, user_id=f1ed594b5cb9438e9328335261e9997f, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:38.769281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1ed5d-8146-41ed-9f18-a8f3dc59b692 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:38.772961 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:38.788904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1200/2395] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:19:38 2022] PUT /identity/v3/projects/476c69341d6b46499a50461862fd8a04/users/f1ed594b5cb9438e9328335261e9997f/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:19:38.800888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31e7bd-c020-46fe-a2a6-390a640c4341 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:38.801196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31e7bd-c020-46fe-a2a6-390a640c4341 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:38.801471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31e7bd-c020-46fe-a2a6-390a640c4341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:38.849884 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-de31e7bd-c020-46fe-a2a6-390a640c4341 None None] MFA Rules not processed for user `f1ed594b5cb9438e9328335261e9997f`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:38.851257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-de31e7bd-c020-46fe-a2a6-390a640c4341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:38.953016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:39.026413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1196/2396] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:19:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:48.793830 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:49.021100 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:19:49.576312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1ed5d-8146-41ed-9f18-a8f3dc59b692 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:49.578205 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-40d1ed5d-8146-41ed-9f18-a8f3dc59b692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:49.652363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1ed5d-8146-41ed-9f18-a8f3dc59b692 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:49.656185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0700e8-dfdc-45d7-a247-79af1be7cd5b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f82adce75b44c6baf85b6b96dd0728', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:49.656794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0700e8-dfdc-45d7-a247-79af1be7cd5b service glance] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:49.657117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0700e8-dfdc-45d7-a247-79af1be7cd5b service glance] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:49.657528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0700e8-dfdc-45d7-a247-79af1be7cd5b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:49.667012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0700e8-dfdc-45d7-a247-79af1be7cd5b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:49.668642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0700e8-dfdc-45d7-a247-79af1be7cd5b service glance] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:49.681286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1201/2397] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:19:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:19:52.178670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6805bc2-16d6-40de-9f86-4640ea5d148b None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:52.179644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6805bc2-16d6-40de-9f86-4640ea5d148b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:52.180399 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6805bc2-16d6-40de-9f86-4640ea5d148b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:52.214729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-c6805bc2-16d6-40de-9f86-4640ea5d148b None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:19:52.217181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c6805bc2-16d6-40de-9f86-4640ea5d148b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:52.311618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1197/2398] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:19:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:19:52.328105 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0700e8-dfdc-45d7-a247-79af1be7cd5b service glance] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:19:52.329609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3c0700e8-dfdc-45d7-a247-79af1be7cd5b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:19:52.408479 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0700e8-dfdc-45d7-a247-79af1be7cd5b service glance] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:19:52.414425 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352af11-cde5-4550-acb0-948e3bebe6cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:19:52.415091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352af11-cde5-4550-acb0-948e3bebe6cb service swift] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:19:52.415405 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352af11-cde5-4550-acb0-948e3bebe6cb service swift] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:19:52.415681 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352af11-cde5-4550-acb0-948e3bebe6cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:19:52.426031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352af11-cde5-4550-acb0-948e3bebe6cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:19:52.427838 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352af11-cde5-4550-acb0-948e3bebe6cb service swift] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:19:52.436943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1202/2399] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:19:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 13:20:02.298018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=25 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:02.433409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=58 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:08.009893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64907451-6248-440a-80d0-18152c6652be None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:08.010678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64907451-6248-440a-80d0-18152c6652be None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:08.010678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64907451-6248-440a-80d0-18152c6652be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:08.031188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-64907451-6248-440a-80d0-18152c6652be None None] MFA Rules not processed for user `406bdf61f9284ebc8620b21ee414ec91`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:08.032422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-64907451-6248-440a-80d0-18152c6652be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:08.101081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1198/2400] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:20:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:14.702912 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0128d-2912-4a35-8659-249b2bd3bc55 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:14.703449 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0128d-2912-4a35-8659-249b2bd3bc55 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:14.703565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0128d-2912-4a35-8659-249b2bd3bc55 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:14.704986 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1203/2401] 10.222.0.61 () {54 vars in 961 bytes} [Wed Sep 7 13:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:20:14.710090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddabf1-a416-460c-9f1f-b6c4536df2dd None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:14.710518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddabf1-a416-460c-9f1f-b6c4536df2dd None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:14.710638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddabf1-a416-460c-9f1f-b6c4536df2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:14.732213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-7dddabf1-a416-460c-9f1f-b6c4536df2dd None None] MFA Rules not processed for user `9e283e62741f40439d02e545a212807a`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:14.733624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7dddabf1-a416-460c-9f1f-b6c4536df2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:14.798653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1199/2402] 10.222.0.61 () {58 vars in 1058 bytes} [Wed Sep 7 13:20:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:14.806363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0128d-2912-4a35-8659-249b2bd3bc55 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:14.820723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0128d-2912-4a35-8659-249b2bd3bc55 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:14.825813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a9068-1c85-4b87-b800-6976d8b15dfc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:14.826414 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a9068-1c85-4b87-b800-6976d8b15dfc service placement] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:14.826691 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a9068-1c85-4b87-b800-6976d8b15dfc service placement] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:14.826982 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a9068-1c85-4b87-b800-6976d8b15dfc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:14.829692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-262a9068-1c85-4b87-b800-6976d8b15dfc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:14.843448 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a9068-1c85-4b87-b800-6976d8b15dfc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:14.845035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a9068-1c85-4b87-b800-6976d8b15dfc service placement] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:14.855598 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1204/2403] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:20:16.566833 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb4afb-fca8-447b-90a4-126d359a5f04 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:16.567693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb4afb-fca8-447b-90a4-126d359a5f04 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:16.567693 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb4afb-fca8-447b-90a4-126d359a5f04 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:16.568981 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1200/2404] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:20:16.582983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f54a6-0760-4cf5-a143-35962f402455 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:16.583593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f54a6-0760-4cf5-a143-35962f402455 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:16.584135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f54a6-0760-4cf5-a143-35962f402455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:16.612247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-202f54a6-0760-4cf5-a143-35962f402455 None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:16.613497 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-202f54a6-0760-4cf5-a143-35962f402455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:16.707745 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1205/2405] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:20:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:20:24.790185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=54 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:26.695911 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=37 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:28.496136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb4afb-fca8-447b-90a4-126d359a5f04 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:28.498592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6ffb4afb-fca8-447b-90a4-126d359a5f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:28.513448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb4afb-fca8-447b-90a4-126d359a5f04 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:28.518128 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f92444-8cbe-4a81-9346-ee740752487d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f82adce75b44c6baf85b6b96dd0728', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:28.518753 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f92444-8cbe-4a81-9346-ee740752487d service glance] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:28.519034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f92444-8cbe-4a81-9346-ee740752487d service glance] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:28.519355 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f92444-8cbe-4a81-9346-ee740752487d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:28.530985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f92444-8cbe-4a81-9346-ee740752487d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:28.532491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f92444-8cbe-4a81-9346-ee740752487d service glance] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:28.545543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1201/2406] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:20:28.605383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949c4af-955e-45a3-99d9-fb6f0aab622a None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:28.606017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949c4af-955e-45a3-99d9-fb6f0aab622a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:28.606017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949c4af-955e-45a3-99d9-fb6f0aab622a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:28.635090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a949c4af-955e-45a3-99d9-fb6f0aab622a None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:28.636976 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a949c4af-955e-45a3-99d9-fb6f0aab622a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:28.647019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1206/2407] 10.222.0.61 () {58 vars in 967 bytes} [Wed Sep 7 13:20:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:28.684982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f92444-8cbe-4a81-9346-ee740752487d service glance] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:28.697755 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f92444-8cbe-4a81-9346-ee740752487d service glance] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:28.705533 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52846be-dc9f-4830-b0a8-2e26318ce027 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:28.706483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52846be-dc9f-4830-b0a8-2e26318ce027 service swift] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:28.706797 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52846be-dc9f-4830-b0a8-2e26318ce027 service swift] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:28.707072 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52846be-dc9f-4830-b0a8-2e26318ce027 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:28.717219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52846be-dc9f-4830-b0a8-2e26318ce027 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:28.718956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52846be-dc9f-4830-b0a8-2e26318ce027 service swift] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:28.730417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1202/2408] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:20:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:20:29.263886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514cbff-523c-412d-a9ba-3f64880aaf2c None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:29.264346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514cbff-523c-412d-a9ba-3f64880aaf2c None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:29.264840 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514cbff-523c-412d-a9ba-3f64880aaf2c None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:29.268148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1207/2409] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:20:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:20:29.273489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592076d-74bb-4f3f-ad68-745feeeb9b9e None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:29.273849 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592076d-74bb-4f3f-ad68-745feeeb9b9e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:29.274134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592076d-74bb-4f3f-ad68-745feeeb9b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:29.307397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-0592076d-74bb-4f3f-ad68-745feeeb9b9e None None] MFA Rules not processed for user `de05c08e30dd4fff93f1c97aac9553d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:29.309586 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-0592076d-74bb-4f3f-ad68-745feeeb9b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:29.400817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1203/2410] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:20:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:29.415945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514cbff-523c-412d-a9ba-3f64880aaf2c None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:29.417869 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b514cbff-523c-412d-a9ba-3f64880aaf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:29.436395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514cbff-523c-412d-a9ba-3f64880aaf2c None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:29.440323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d760c-c9dc-4757-a46e-6c3ecd9a9021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:29.440969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d760c-c9dc-4757-a46e-6c3ecd9a9021 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:29.441245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d760c-c9dc-4757-a46e-6c3ecd9a9021 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:29.441521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d760c-c9dc-4757-a46e-6c3ecd9a9021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:29.458532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d760c-c9dc-4757-a46e-6c3ecd9a9021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:29.459924 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d760c-c9dc-4757-a46e-6c3ecd9a9021 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:29.470308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1208/2411] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:20:29.592432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b23f8-0dde-40ef-ab3f-86a62e3a88cc None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:29.593038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b23f8-0dde-40ef-ab3f-86a62e3a88cc None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:29.593435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b23f8-0dde-40ef-ab3f-86a62e3a88cc None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:29.594937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1204/2412] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:20:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:20:29.603069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9f9f6-7c3d-44ab-869c-8f58475cb5c1 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:29.603393 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9f9f6-7c3d-44ab-869c-8f58475cb5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:29.603670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9f9f6-7c3d-44ab-869c-8f58475cb5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:29.631821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-fff9f9f6-7c3d-44ab-869c-8f58475cb5c1 None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:29.633451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-fff9f9f6-7c3d-44ab-869c-8f58475cb5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:29.648287 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1209/2413] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:20:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:29.660759 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b23f8-0dde-40ef-ab3f-86a62e3a88cc None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:29.681317 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b23f8-0dde-40ef-ab3f-86a62e3a88cc None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:29.685801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556823-3ea8-4f98-a3ff-28df0fcc7c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:29.686431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556823-3ea8-4f98-a3ff-28df0fcc7c8b service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:29.686710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556823-3ea8-4f98-a3ff-28df0fcc7c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:29.686981 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556823-3ea8-4f98-a3ff-28df0fcc7c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:29.704062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8556823-3ea8-4f98-a3ff-28df0fcc7c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:29.705453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8556823-3ea8-4f98-a3ff-28df0fcc7c8b service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:29.718338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1205/2414] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:20:34.162280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9f9f6-7c3d-44ab-869c-8f58475cb5c1 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:34.166174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-fff9f9f6-7c3d-44ab-869c-8f58475cb5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:34.276883 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9f9f6-7c3d-44ab-869c-8f58475cb5c1 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:34.283355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17079a04-db82-4616-b8fb-3e9d564c1198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:34.284438 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17079a04-db82-4616-b8fb-3e9d564c1198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:34.284959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17079a04-db82-4616-b8fb-3e9d564c1198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:34.285446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17079a04-db82-4616-b8fb-3e9d564c1198 admin admin] PATH_INFO: `/v3/users/3f93418e3e264a1f8597f789ee888b4e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:34.288357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17079a04-db82-4616-b8fb-3e9d564c1198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f93418e3e264a1f8597f789ee888b4e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:34.291536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17079a04-db82-4616-b8fb-3e9d564c1198 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:34.376044 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-17079a04-db82-4616-b8fb-3e9d564c1198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f93418e3e264a1f8597f789ee888b4e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:34.418981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-17079a04-db82-4616-b8fb-3e9d564c1198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f93418e3e264a1f8597f789ee888b4e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:34.430555 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-17079a04-db82-4616-b8fb-3e9d564c1198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f93418e3e264a1f8597f789ee888b4e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:34.431661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-17079a04-db82-4616-b8fb-3e9d564c1198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f93418e3e264a1f8597f789ee888b4e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:34.435766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-17079a04-db82-4616-b8fb-3e9d564c1198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f93418e3e264a1f8597f789ee888b4e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:34.467024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1210/2415] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:20:34 2022] DELETE /identity/v3/users/3f93418e3e264a1f8597f789ee888b4e => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:34.690224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556823-3ea8-4f98-a3ff-28df0fcc7c8b service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:34.703509 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556823-3ea8-4f98-a3ff-28df0fcc7c8b service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:34.776021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03925c35-4024-444a-b6a4-7eff47d4b6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:34.778171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03925c35-4024-444a-b6a4-7eff47d4b6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:34.778433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03925c35-4024-444a-b6a4-7eff47d4b6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:34.778831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03925c35-4024-444a-b6a4-7eff47d4b6f4 admin admin] PATH_INFO: `/v3/projects/6d3b344911654d3a83328d54b8feca0e` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:34.781537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03925c35-4024-444a-b6a4-7eff47d4b6f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d3b344911654d3a83328d54b8feca0e)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:34.783752 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03925c35-4024-444a-b6a4-7eff47d4b6f4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:34.850603 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-03925c35-4024-444a-b6a4-7eff47d4b6f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d3b344911654d3a83328d54b8feca0e', 'request_id': 'req-03925c35-4024-444a-b6a4-7eff47d4b6f4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:34.873296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-03925c35-4024-444a-b6a4-7eff47d4b6f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d3b344911654d3a83328d54b8feca0e', 'request_id': 'req-03925c35-4024-444a-b6a4-7eff47d4b6f4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:34.882080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-03925c35-4024-444a-b6a4-7eff47d4b6f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d3b344911654d3a83328d54b8feca0e', 'request_id': 'req-03925c35-4024-444a-b6a4-7eff47d4b6f4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:34.899836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-03925c35-4024-444a-b6a4-7eff47d4b6f4 admin admin] The token cache is being invalidate because project 6d3b344911654d3a83328d54b8feca0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:20:34.900536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:34.901620 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-03925c35-4024-444a-b6a4-7eff47d4b6f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:34.902979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-03925c35-4024-444a-b6a4-7eff47d4b6f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:20:34.905025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1206/2416] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:20:34 2022] DELETE /identity/v3/projects/6d3b344911654d3a83328d54b8feca0e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:34.911088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3ba7c-2bc2-4f38-a745-82b527dcaacb None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:34.911598 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3ba7c-2bc2-4f38-a745-82b527dcaacb None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:34.911966 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3ba7c-2bc2-4f38-a745-82b527dcaacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:34.951933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a4a3ba7c-2bc2-4f38-a745-82b527dcaacb None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:34.952983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a4a3ba7c-2bc2-4f38-a745-82b527dcaacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:35.047378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:35.079494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1211/2417] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:35.088241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620449bf-67fb-418e-8518-7a3466ffaac6 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:35.088524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620449bf-67fb-418e-8518-7a3466ffaac6 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:35.088796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620449bf-67fb-418e-8518-7a3466ffaac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:35.116508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-620449bf-67fb-418e-8518-7a3466ffaac6 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:35.117726 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-620449bf-67fb-418e-8518-7a3466ffaac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:35.130315 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1207/2418] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:35.137958 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3ba7c-2bc2-4f38-a745-82b527dcaacb None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:35.153934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3ba7c-2bc2-4f38-a745-82b527dcaacb None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:35.160802 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc489282-5b8b-4348-944e-b205e2e13b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:35.162973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc489282-5b8b-4348-944e-b205e2e13b45 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:35.163622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc489282-5b8b-4348-944e-b205e2e13b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:35.167286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc489282-5b8b-4348-944e-b205e2e13b45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:35.168296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc489282-5b8b-4348-944e-b205e2e13b45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:20:35.168843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc489282-5b8b-4348-944e-b205e2e13b45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:35.175494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc489282-5b8b-4348-944e-b205e2e13b45 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:35.203824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1212/2419] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:20:35.210946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620449bf-67fb-418e-8518-7a3466ffaac6 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:35.223134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620449bf-67fb-418e-8518-7a3466ffaac6 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:35.228687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e0195-b59e-45b4-892c-f9552cff2e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:35.229678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e0195-b59e-45b4-892c-f9552cff2e53 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:35.230098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e0195-b59e-45b4-892c-f9552cff2e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:35.230533 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e0195-b59e-45b4-892c-f9552cff2e53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:35.232028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e0195-b59e-45b4-892c-f9552cff2e53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:35.235314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e0195-b59e-45b4-892c-f9552cff2e53 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:35.270328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1208/2420] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:20:35 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:20:35.275042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc489282-5b8b-4348-944e-b205e2e13b45 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:35.291318 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc489282-5b8b-4348-944e-b205e2e13b45 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:35.381383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293531c-7ba6-4afb-af5f-59ad002dbfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:35.382089 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293531c-7ba6-4afb-af5f-59ad002dbfcd admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:35.382443 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293531c-7ba6-4afb-af5f-59ad002dbfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:35.382788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293531c-7ba6-4afb-af5f-59ad002dbfcd admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:35.383811 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b293531c-7ba6-4afb-af5f-59ad002dbfcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:35.385848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b293531c-7ba6-4afb-af5f-59ad002dbfcd admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:35.413624 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1213/2421] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:20:35 2022] POST /identity/v3/users => generated 283 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:20:35.418397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e0195-b59e-45b4-892c-f9552cff2e53 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:35.433233 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e0195-b59e-45b4-892c-f9552cff2e53 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:35.440524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45d4ac-a706-4286-929b-9da499004ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:35.441480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45d4ac-a706-4286-929b-9da499004ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:35.441911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45d4ac-a706-4286-929b-9da499004ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:35.442457 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45d4ac-a706-4286-929b-9da499004ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:35.443072 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45d4ac-a706-4286-929b-9da499004ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:35.445263 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45d4ac-a706-4286-929b-9da499004ca1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:35.465956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1209/2422] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:20:35.472720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293531c-7ba6-4afb-af5f-59ad002dbfcd admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:35.498594 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293531c-7ba6-4afb-af5f-59ad002dbfcd admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:35.505785 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10076940-1fa4-4edd-bb37-e5c9bd8d5cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:35.506233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10076940-1fa4-4edd-bb37-e5c9bd8d5cde admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:35.506675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10076940-1fa4-4edd-bb37-e5c9bd8d5cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:35.507047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10076940-1fa4-4edd-bb37-e5c9bd8d5cde admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:35.507943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10076940-1fa4-4edd-bb37-e5c9bd8d5cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:35.509772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10076940-1fa4-4edd-bb37-e5c9bd8d5cde admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:35.528041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1214/2423] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:20:35.535755 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45d4ac-a706-4286-929b-9da499004ca1 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:35.556098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45d4ac-a706-4286-929b-9da499004ca1 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:35.563085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70256f-9994-4df5-bac3-a0feef0dd24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:35.564617 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70256f-9994-4df5-bac3-a0feef0dd24d admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:35.564971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70256f-9994-4df5-bac3-a0feef0dd24d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:35.565447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70256f-9994-4df5-bac3-a0feef0dd24d admin admin] PATH_INFO: `/v3/projects/6f70d5c228bb4b30a50e909e1c6de886/users/72598e8009b8494696bbaaa992bba4fe/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:35.585780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70256f-9994-4df5-bac3-a0feef0dd24d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f70d5c228bb4b30a50e909e1c6de886, user_id=72598e8009b8494696bbaaa992bba4fe, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:35.590973 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70256f-9994-4df5-bac3-a0feef0dd24d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:35.627959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09485d-955f-44fe-b8b7-5ff9b044b396 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:35.628441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09485d-955f-44fe-b8b7-5ff9b044b396 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:35.628902 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09485d-955f-44fe-b8b7-5ff9b044b396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:35.632968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1210/2424] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:20:35 2022] PUT /identity/v3/projects/6f70d5c228bb4b30a50e909e1c6de886/users/72598e8009b8494696bbaaa992bba4fe/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:20:35.682716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-cf09485d-955f-44fe-b8b7-5ff9b044b396 None None] MFA Rules not processed for user `72598e8009b8494696bbaaa992bba4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:35.684005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-cf09485d-955f-44fe-b8b7-5ff9b044b396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:35.772670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1215/2425] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:20:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:20:35.784641 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70256f-9994-4df5-bac3-a0feef0dd24d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:35.787069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-9f70256f-9994-4df5-bac3-a0feef0dd24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:35.862968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70256f-9994-4df5-bac3-a0feef0dd24d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:35.868552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8319188-5af5-4dcc-819f-73ae2bdee2f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:35.869341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8319188-5af5-4dcc-819f-73ae2bdee2f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:35.869622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8319188-5af5-4dcc-819f-73ae2bdee2f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:35.869895 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8319188-5af5-4dcc-819f-73ae2bdee2f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:35.881434 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8319188-5af5-4dcc-819f-73ae2bdee2f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:35.883119 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8319188-5af5-4dcc-819f-73ae2bdee2f0 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:35.894656 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1211/2426] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:20:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:20:40.209673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f6e6b-2b7e-4c2a-8b18-ec24c7002ef7 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:40.209673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f6e6b-2b7e-4c2a-8b18-ec24c7002ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:40.209673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f6e6b-2b7e-4c2a-8b18-ec24c7002ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:40.236982 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f33f6e6b-2b7e-4c2a-8b18-ec24c7002ef7 None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:40.238850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f33f6e6b-2b7e-4c2a-8b18-ec24c7002ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:40.308323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:40.325126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1216/2427] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:20:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:20:40.342982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8319188-5af5-4dcc-819f-73ae2bdee2f0 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:20:40.344719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c8319188-5af5-4dcc-819f-73ae2bdee2f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:40.453764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8319188-5af5-4dcc-819f-73ae2bdee2f0 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:20:40.459568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ab4ed-05af-4a33-8981-12e4c8c21042 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:20:40.460550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ab4ed-05af-4a33-8981-12e4c8c21042 service swift] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:40.460993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ab4ed-05af-4a33-8981-12e4c8c21042 service swift] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:40.461537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ab4ed-05af-4a33-8981-12e4c8c21042 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:40.472291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ab4ed-05af-4a33-8981-12e4c8c21042 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:20:40.473909 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ab4ed-05af-4a33-8981-12e4c8c21042 service swift] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:20:40.485807 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1212/2428] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:20:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:20:50.321452 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:50.482722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=85,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:20:57.844230 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923ab9c-033c-4bc8-a204-dce07570b7fd None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:20:57.844825 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923ab9c-033c-4bc8-a204-dce07570b7fd None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:20:57.844825 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923ab9c-033c-4bc8-a204-dce07570b7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:20:57.866915 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-9923ab9c-033c-4bc8-a204-dce07570b7fd None None] MFA Rules not processed for user `57896ea1ff2046d282fe47e2a8191676`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:20:57.868192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9923ab9c-033c-4bc8-a204-dce07570b7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:20:57.944916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1217/2429] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:20:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:00.201993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ab4ed-05af-4a33-8981-12e4c8c21042 service swift] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:00.204381 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-156ab4ed-05af-4a33-8981-12e4c8c21042 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:00.288892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ab4ed-05af-4a33-8981-12e4c8c21042 service swift] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:00.294221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:00.294977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:00.295364 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:00.295757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] PATH_INFO: `/v3/users/f1ed594b5cb9438e9328335261e9997f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:00.297439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1ed594b5cb9438e9328335261e9997f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:00.300335 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:00.370793 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1ed594b5cb9438e9328335261e9997f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:00.381217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1ed594b5cb9438e9328335261e9997f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:00.383392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1ed594b5cb9438e9328335261e9997f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:00.383392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1ed594b5cb9438e9328335261e9997f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:00.402151 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1ed594b5cb9438e9328335261e9997f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:00.419015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1213/2430] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:21:00 2022] DELETE /identity/v3/users/f1ed594b5cb9438e9328335261e9997f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:00.423899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342c7cd-a41b-430a-9117-9dfc7b9e6101 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:00.424454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342c7cd-a41b-430a-9117-9dfc7b9e6101 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:00.424454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342c7cd-a41b-430a-9117-9dfc7b9e6101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:00.444123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a342c7cd-a41b-430a-9117-9dfc7b9e6101 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:00.445419 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a342c7cd-a41b-430a-9117-9dfc7b9e6101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:00.514908 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1218/2431] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:00.524829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:00.526305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:00.586621 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd605-9782-4b43-8120-33d6272c6b19 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:00.590350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74119d-5852-4a66-8b2d-32b205cf51a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:00.591002 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74119d-5852-4a66-8b2d-32b205cf51a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:00.591312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74119d-5852-4a66-8b2d-32b205cf51a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:00.591596 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74119d-5852-4a66-8b2d-32b205cf51a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:00.602108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74119d-5852-4a66-8b2d-32b205cf51a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:00.603607 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74119d-5852-4a66-8b2d-32b205cf51a9 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:00.613947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1214/2432] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:00.911210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342c7cd-a41b-430a-9117-9dfc7b9e6101 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:00.912663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a342c7cd-a41b-430a-9117-9dfc7b9e6101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:00.926088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342c7cd-a41b-430a-9117-9dfc7b9e6101 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:00.930918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a9a19-1835-47e7-9c46-560e336230e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:00.931574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a9a19-1835-47e7-9c46-560e336230e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:00.931858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a9a19-1835-47e7-9c46-560e336230e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:00.932125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a9a19-1835-47e7-9c46-560e336230e9 admin admin] PATH_INFO: `/v3/projects/476c69341d6b46499a50461862fd8a04` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:00.934041 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a9a19-1835-47e7-9c46-560e336230e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=476c69341d6b46499a50461862fd8a04)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:00.935980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a9a19-1835-47e7-9c46-560e336230e9 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:00.986715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c06a9a19-1835-47e7-9c46-560e336230e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '476c69341d6b46499a50461862fd8a04', 'request_id': 'req-c06a9a19-1835-47e7-9c46-560e336230e9'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:00.993546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c06a9a19-1835-47e7-9c46-560e336230e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '476c69341d6b46499a50461862fd8a04', 'request_id': 'req-c06a9a19-1835-47e7-9c46-560e336230e9'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:01.000128 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c06a9a19-1835-47e7-9c46-560e336230e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '476c69341d6b46499a50461862fd8a04', 'request_id': 'req-c06a9a19-1835-47e7-9c46-560e336230e9'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:01.012347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c06a9a19-1835-47e7-9c46-560e336230e9 admin admin] The token cache is being invalidate because project 476c69341d6b46499a50461862fd8a04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:21:01.012718 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c06a9a19-1835-47e7-9c46-560e336230e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:01.013302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c06a9a19-1835-47e7-9c46-560e336230e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:01.014881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1219/2433] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:21:00 2022] DELETE /identity/v3/projects/476c69341d6b46499a50461862fd8a04 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:01.019875 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27628b18-0c93-4ba0-9546-9b4ad0ace45c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:01.020227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27628b18-0c93-4ba0-9546-9b4ad0ace45c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:01.020571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27628b18-0c93-4ba0-9546-9b4ad0ace45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:01.041664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-27628b18-0c93-4ba0-9546-9b4ad0ace45c None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:01.043583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-27628b18-0c93-4ba0-9546-9b4ad0ace45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:01.065626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:01.116437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1215/2434] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:21:01.124518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee289f-9f98-442b-8ba8-3f89f5306cac None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:01.124900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee289f-9f98-442b-8ba8-3f89f5306cac None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:01.125188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee289f-9f98-442b-8ba8-3f89f5306cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:01.146820 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-c7ee289f-9f98-442b-8ba8-3f89f5306cac None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:01.148117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c7ee289f-9f98-442b-8ba8-3f89f5306cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:01.157078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1220/2435] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:21:01.161274 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27628b18-0c93-4ba0-9546-9b4ad0ace45c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:01.171843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27628b18-0c93-4ba0-9546-9b4ad0ace45c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:01.177646 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f8e07-e665-43d4-84b2-651ef9b13f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:01.178700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f8e07-e665-43d4-84b2-651ef9b13f3b admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:01.179268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f8e07-e665-43d4-84b2-651ef9b13f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:01.179787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f8e07-e665-43d4-84b2-651ef9b13f3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:01.180652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f8e07-e665-43d4-84b2-651ef9b13f3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:21:01.181178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f8e07-e665-43d4-84b2-651ef9b13f3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:01.183329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f8e07-e665-43d4-84b2-651ef9b13f3b admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:01.198729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1216/2436] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:21:01.203540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee289f-9f98-442b-8ba8-3f89f5306cac None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:01.213907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee289f-9f98-442b-8ba8-3f89f5306cac None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:01.219358 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ba1ed-3cad-423d-b674-4eca27b7bd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:01.219979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ba1ed-3cad-423d-b674-4eca27b7bd90 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:01.220271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ba1ed-3cad-423d-b674-4eca27b7bd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:01.220548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ba1ed-3cad-423d-b674-4eca27b7bd90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:01.221207 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ba1ed-3cad-423d-b674-4eca27b7bd90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:01.223148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ba1ed-3cad-423d-b674-4eca27b7bd90 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:01.256099 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:01.267336 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1221/2437] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:21:01 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:21:01.268872 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f8e07-e665-43d4-84b2-651ef9b13f3b admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:01.279467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f8e07-e665-43d4-84b2-651ef9b13f3b admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:01.341467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7878c-de5f-4a43-8455-612b3a17c02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:01.342081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7878c-de5f-4a43-8455-612b3a17c02d admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:01.342363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7878c-de5f-4a43-8455-612b3a17c02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:01.342639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7878c-de5f-4a43-8455-612b3a17c02d admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:01.343324 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7878c-de5f-4a43-8455-612b3a17c02d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:01.345265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7878c-de5f-4a43-8455-612b3a17c02d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:01.367041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1217/2438] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:21:01 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:21:01.372088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ba1ed-3cad-423d-b674-4eca27b7bd90 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:01.382656 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ba1ed-3cad-423d-b674-4eca27b7bd90 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:01.388060 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d5028-4c59-4f09-b76a-d471fdab88cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:01.388666 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d5028-4c59-4f09-b76a-d471fdab88cb admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:01.388948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d5028-4c59-4f09-b76a-d471fdab88cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:01.389222 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d5028-4c59-4f09-b76a-d471fdab88cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:01.389896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d5028-4c59-4f09-b76a-d471fdab88cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:01.391544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d5028-4c59-4f09-b76a-d471fdab88cb admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:01.401874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1222/2439] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:21:01.406003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7878c-de5f-4a43-8455-612b3a17c02d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:01.416301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7878c-de5f-4a43-8455-612b3a17c02d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:01.421017 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385194ae-94b3-4868-8618-bf5a7fae99f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:01.421625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385194ae-94b3-4868-8618-bf5a7fae99f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:01.421938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385194ae-94b3-4868-8618-bf5a7fae99f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:01.422205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385194ae-94b3-4868-8618-bf5a7fae99f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:01.422906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385194ae-94b3-4868-8618-bf5a7fae99f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:01.424618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385194ae-94b3-4868-8618-bf5a7fae99f3 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:01.437244 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1218/2440] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:21:01.441252 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d5028-4c59-4f09-b76a-d471fdab88cb admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:01.453501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d5028-4c59-4f09-b76a-d471fdab88cb admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:01.459184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d473d-a535-4a4d-bfe4-dee420fb920b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:01.459854 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d473d-a535-4a4d-bfe4-dee420fb920b admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:01.460131 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d473d-a535-4a4d-bfe4-dee420fb920b admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:01.460586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d473d-a535-4a4d-bfe4-dee420fb920b admin admin] PATH_INFO: `/v3/projects/b80891c7f16742a3bf07d5481dc4a7d3/users/c1b39cb086ed41d6995dbf4f15074f74/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:01.479903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d473d-a535-4a4d-bfe4-dee420fb920b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b80891c7f16742a3bf07d5481dc4a7d3, user_id=c1b39cb086ed41d6995dbf4f15074f74, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:01.482895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d473d-a535-4a4d-bfe4-dee420fb920b admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:01.500845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1223/2441] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:21:01 2022] PUT /identity/v3/projects/b80891c7f16742a3bf07d5481dc4a7d3/users/c1b39cb086ed41d6995dbf4f15074f74/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:01.574981 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2da1-8607-409d-989d-87f17a33b56d None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:01.575304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2da1-8607-409d-989d-87f17a33b56d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:01.575582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2da1-8607-409d-989d-87f17a33b56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:01.614284 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-60fd2da1-8607-409d-989d-87f17a33b56d None None] MFA Rules not processed for user `c1b39cb086ed41d6995dbf4f15074f74`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:01.616380 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-60fd2da1-8607-409d-989d-87f17a33b56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:01.715027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1219/2442] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:21:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:01.728048 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d473d-a535-4a4d-bfe4-dee420fb920b admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:01.732051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-180d473d-a535-4a4d-bfe4-dee420fb920b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:01.805371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d473d-a535-4a4d-bfe4-dee420fb920b admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:01.811673 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ef836-c25f-4419-8c2b-0a9fbc95a128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:01.812399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ef836-c25f-4419-8c2b-0a9fbc95a128 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:01.812741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ef836-c25f-4419-8c2b-0a9fbc95a128 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:01.813216 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ef836-c25f-4419-8c2b-0a9fbc95a128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:01.828295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ef836-c25f-4419-8c2b-0a9fbc95a128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:01.833422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ef836-c25f-4419-8c2b-0a9fbc95a128 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:01.846469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1224/2443] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:01.879782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd2da1-8607-409d-989d-87f17a33b56d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:01.910798 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:02.005095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd2da1-8607-409d-989d-87f17a33b56d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:02.023148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:02.023818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:02.024104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:02.024384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] PATH_INFO: `/v3/users/c1b39cb086ed41d6995dbf4f15074f74` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:02.026799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1b39cb086ed41d6995dbf4f15074f74)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:02.034367 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:02.136897 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1b39cb086ed41d6995dbf4f15074f74'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:02.151719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1b39cb086ed41d6995dbf4f15074f74'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:02.153179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1b39cb086ed41d6995dbf4f15074f74'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:02.154482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1b39cb086ed41d6995dbf4f15074f74'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:02.179904 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1b39cb086ed41d6995dbf4f15074f74'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:02.207889 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1220/2444] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:21:01 2022] DELETE /identity/v3/users/c1b39cb086ed41d6995dbf4f15074f74 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:02.212179 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f8043-4873-4e37-b36d-4a483cf7952d None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:02.212470 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f8043-4873-4e37-b36d-4a483cf7952d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:02.212751 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f8043-4873-4e37-b36d-4a483cf7952d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:02.251799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-ee8f8043-4873-4e37-b36d-4a483cf7952d None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:02.251944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ee8f8043-4873-4e37-b36d-4a483cf7952d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:02.350150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1225/2445] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:21:02.365800 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:02.367418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:02.449905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57abf64-ddea-4a61-82d3-41f826fbc27c admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:02.454349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc4740-14c6-4698-bdea-6e4f4bbce4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:02.454976 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4740-14c6-4698-bdea-6e4f4bbce4cb service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:02.455293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4740-14c6-4698-bdea-6e4f4bbce4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:02.455572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4740-14c6-4698-bdea-6e4f4bbce4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:02.466339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc4740-14c6-4698-bdea-6e4f4bbce4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:02.467826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc4740-14c6-4698-bdea-6e4f4bbce4cb service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:02.479418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1221/2446] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:02.778503 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f8043-4873-4e37-b36d-4a483cf7952d None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:02.780716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ee8f8043-4873-4e37-b36d-4a483cf7952d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:02.793956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f8043-4873-4e37-b36d-4a483cf7952d None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:02.801672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:02.802424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:02.802702 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:02.802973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] PATH_INFO: `/v3/projects/b80891c7f16742a3bf07d5481dc4a7d3` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:02.804661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b80891c7f16742a3bf07d5481dc4a7d3)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:02.806613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:02.878827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b80891c7f16742a3bf07d5481dc4a7d3', 'request_id': 'req-4ef664bf-7f83-4078-b82c-583d6562c1f3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:02.887345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b80891c7f16742a3bf07d5481dc4a7d3', 'request_id': 'req-4ef664bf-7f83-4078-b82c-583d6562c1f3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:02.899622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b80891c7f16742a3bf07d5481dc4a7d3', 'request_id': 'req-4ef664bf-7f83-4078-b82c-583d6562c1f3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:02.902933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:02.922904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] The token cache is being invalidate because project b80891c7f16742a3bf07d5481dc4a7d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:21:02.923503 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:02.924241 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:02.925864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1226/2447] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:21:02 2022] DELETE /identity/v3/projects/b80891c7f16742a3bf07d5481dc4a7d3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:09.705225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3790ded-d7f0-4a54-938c-e0899a2cf847 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:09.706405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3790ded-d7f0-4a54-938c-e0899a2cf847 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:09.706913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3790ded-d7f0-4a54-938c-e0899a2cf847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:09.743196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-f3790ded-d7f0-4a54-938c-e0899a2cf847 None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:09.743196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f3790ded-d7f0-4a54-938c-e0899a2cf847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:09.786896 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:09.821557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1222/2448] 10.222.0.61 () {58 vars in 967 bytes} [Wed Sep 7 13:21:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:09.843154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:09.845541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:09.930638 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef664bf-7f83-4078-b82c-583d6562c1f3 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:09.937130 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589ec11-63fe-46b7-8e97-35f5988c8c01 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:09.937768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589ec11-63fe-46b7-8e97-35f5988c8c01 service swift] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:09.938052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589ec11-63fe-46b7-8e97-35f5988c8c01 service swift] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:09.938327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589ec11-63fe-46b7-8e97-35f5988c8c01 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:09.949663 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589ec11-63fe-46b7-8e97-35f5988c8c01 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:09.951390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589ec11-63fe-46b7-8e97-35f5988c8c01 service swift] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:09.977370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1227/2449] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:21:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:21:19.816823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=7 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:19.960676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=34 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:23.856497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3790ded-d7f0-4a54-938c-e0899a2cf847 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:23.858266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-f3790ded-d7f0-4a54-938c-e0899a2cf847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:23.921619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3790ded-d7f0-4a54-938c-e0899a2cf847 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:23.925426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ae2b0-791f-4956-bd72-1923243530c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:23.926071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ae2b0-791f-4956-bd72-1923243530c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:23.926345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ae2b0-791f-4956-bd72-1923243530c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:23.926617 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ae2b0-791f-4956-bd72-1923243530c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:23.928358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-271ae2b0-791f-4956-bd72-1923243530c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:24.018327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ae2b0-791f-4956-bd72-1923243530c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:24.021173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ae2b0-791f-4956-bd72-1923243530c7 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:24.038861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1223/2450] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:25.135546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87b868-c161-467d-a935-9e7b05288329 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:25.136127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87b868-c161-467d-a935-9e7b05288329 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:25.136230 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87b868-c161-467d-a935-9e7b05288329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:25.156627 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-2c87b868-c161-467d-a935-9e7b05288329 None None] MFA Rules not processed for user `d6083ed15f884ffea8550f28addcbfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:25.157996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2c87b868-c161-467d-a935-9e7b05288329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:25.230368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1228/2451] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:21:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:25.242994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ae2b0-791f-4956-bd72-1923243530c7 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:25.256913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ae2b0-791f-4956-bd72-1923243530c7 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:25.260802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10913e9f-6c33-41ba-8176-bb2542d26e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:25.261453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10913e9f-6c33-41ba-8176-bb2542d26e0d service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:25.261770 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10913e9f-6c33-41ba-8176-bb2542d26e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:25.262039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10913e9f-6c33-41ba-8176-bb2542d26e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:25.264349 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-10913e9f-6c33-41ba-8176-bb2542d26e0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:25.283369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10913e9f-6c33-41ba-8176-bb2542d26e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:25.284860 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10913e9f-6c33-41ba-8176-bb2542d26e0d service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:25.299138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1224/2452] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:35.221570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=25 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:35.292646 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=60 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:35.301299 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87b868-c161-467d-a935-9e7b05288329 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:35.302659 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2c87b868-c161-467d-a935-9e7b05288329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:35.316070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87b868-c161-467d-a935-9e7b05288329 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:35.320259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea428a7f-a9ca-4601-a677-1a554ef1f88a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:35.320880 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea428a7f-a9ca-4601-a677-1a554ef1f88a service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:35.321158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea428a7f-a9ca-4601-a677-1a554ef1f88a service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:35.321431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea428a7f-a9ca-4601-a677-1a554ef1f88a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:35.324460 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ea428a7f-a9ca-4601-a677-1a554ef1f88a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:35.399474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea428a7f-a9ca-4601-a677-1a554ef1f88a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:35.400956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea428a7f-a9ca-4601-a677-1a554ef1f88a service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:35.410530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1229/2453] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:37.099058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10913e9f-6c33-41ba-8176-bb2542d26e0d service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:37.101013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-10913e9f-6c33-41ba-8176-bb2542d26e0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:37.203912 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10913e9f-6c33-41ba-8176-bb2542d26e0d service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:37.211389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:37.212449 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:37.212934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:37.213789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] PATH_INFO: `/v3/users/72598e8009b8494696bbaaa992bba4fe` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:37.215004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72598e8009b8494696bbaaa992bba4fe)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:37.217033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:37.293625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72598e8009b8494696bbaaa992bba4fe'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:37.304480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72598e8009b8494696bbaaa992bba4fe'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:37.305409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72598e8009b8494696bbaaa992bba4fe'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:37.306172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72598e8009b8494696bbaaa992bba4fe'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:37.325059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72598e8009b8494696bbaaa992bba4fe'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:37.340845 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1225/2454] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:21:37 2022] DELETE /identity/v3/users/72598e8009b8494696bbaaa992bba4fe => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:21:37.345748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551519d-db01-47ba-826c-91b7b54f623f None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:37.346332 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551519d-db01-47ba-826c-91b7b54f623f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:37.346332 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551519d-db01-47ba-826c-91b7b54f623f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:37.369544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-0551519d-db01-47ba-826c-91b7b54f623f None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:37.371214 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-0551519d-db01-47ba-826c-91b7b54f623f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:37.445019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1230/2455] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:21:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:37.455859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:37.457991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:37.525091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47703f28-f697-4bd4-9e08-56e07f3dd9b5 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:37.530252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db6e85-a067-418a-844e-f64f8f13a44a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:37.531316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db6e85-a067-418a-844e-f64f8f13a44a service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:37.531808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db6e85-a067-418a-844e-f64f8f13a44a service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:37.532270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db6e85-a067-418a-844e-f64f8f13a44a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:37.546450 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db6e85-a067-418a-844e-f64f8f13a44a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:37.548167 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db6e85-a067-418a-844e-f64f8f13a44a service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:37.562978 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1226/2456] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:37.984072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551519d-db01-47ba-826c-91b7b54f623f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:37.985911 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-0551519d-db01-47ba-826c-91b7b54f623f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:38.006653 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551519d-db01-47ba-826c-91b7b54f623f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:38.012892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be17d8-21b3-4bb1-a20f-956a07115952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:38.014277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be17d8-21b3-4bb1-a20f-956a07115952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:38.014934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be17d8-21b3-4bb1-a20f-956a07115952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:38.016853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be17d8-21b3-4bb1-a20f-956a07115952 admin admin] PATH_INFO: `/v3/projects/6f70d5c228bb4b30a50e909e1c6de886` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:38.019750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be17d8-21b3-4bb1-a20f-956a07115952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f70d5c228bb4b30a50e909e1c6de886)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:38.023974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be17d8-21b3-4bb1-a20f-956a07115952 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:38.096754 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f2be17d8-21b3-4bb1-a20f-956a07115952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f70d5c228bb4b30a50e909e1c6de886', 'request_id': 'req-f2be17d8-21b3-4bb1-a20f-956a07115952'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:38.107024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f2be17d8-21b3-4bb1-a20f-956a07115952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f70d5c228bb4b30a50e909e1c6de886', 'request_id': 'req-f2be17d8-21b3-4bb1-a20f-956a07115952'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:38.114211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f2be17d8-21b3-4bb1-a20f-956a07115952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f70d5c228bb4b30a50e909e1c6de886', 'request_id': 'req-f2be17d8-21b3-4bb1-a20f-956a07115952'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:38.140082 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f2be17d8-21b3-4bb1-a20f-956a07115952 admin admin] The token cache is being invalidate because project 6f70d5c228bb4b30a50e909e1c6de886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:21:38.140531 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f2be17d8-21b3-4bb1-a20f-956a07115952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:38.141741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f2be17d8-21b3-4bb1-a20f-956a07115952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:38.144835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1231/2457] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:21:37 2022] DELETE /identity/v3/projects/6f70d5c228bb4b30a50e909e1c6de886 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:38.149453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5d50f-035b-4832-bc17-94f79ab70350 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:38.149994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5d50f-035b-4832-bc17-94f79ab70350 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:38.149994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5d50f-035b-4832-bc17-94f79ab70350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:38.172437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-d8b5d50f-035b-4832-bc17-94f79ab70350 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:38.173853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d8b5d50f-035b-4832-bc17-94f79ab70350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:38.199354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:38.255543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1227/2458] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:38.328181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2b7d0-1e60-4566-8bff-24c5305edfff None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:38.329693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2b7d0-1e60-4566-8bff-24c5305edfff None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:38.330353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2b7d0-1e60-4566-8bff-24c5305edfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:38.357726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-3ed2b7d0-1e60-4566-8bff-24c5305edfff None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:38.359856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3ed2b7d0-1e60-4566-8bff-24c5305edfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:38.370102 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1232/2459] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:38.374509 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5d50f-035b-4832-bc17-94f79ab70350 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:38.385260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5d50f-035b-4832-bc17-94f79ab70350 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:38.392534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbeba87-249d-4211-8874-4b5111f67fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:38.393655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbeba87-249d-4211-8874-4b5111f67fab admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:38.394121 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbeba87-249d-4211-8874-4b5111f67fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:38.394459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbeba87-249d-4211-8874-4b5111f67fab admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:38.395571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbeba87-249d-4211-8874-4b5111f67fab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:21:38.396368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbeba87-249d-4211-8874-4b5111f67fab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:38.398465 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbeba87-249d-4211-8874-4b5111f67fab admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:38.412974 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1228/2460] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:21:38.418201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2b7d0-1e60-4566-8bff-24c5305edfff None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:38.430182 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2b7d0-1e60-4566-8bff-24c5305edfff None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:38.436986 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3f3d2-501a-4d7e-8240-202671522d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:38.437847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3f3d2-501a-4d7e-8240-202671522d1a admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:38.438115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3f3d2-501a-4d7e-8240-202671522d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:38.438392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3f3d2-501a-4d7e-8240-202671522d1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:38.440553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3f3d2-501a-4d7e-8240-202671522d1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:38.441841 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3f3d2-501a-4d7e-8240-202671522d1a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:38.460495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:38.498426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1233/2461] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:21:38 2022] POST /identity/v3/projects => generated 399 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:21:38.502957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbeba87-249d-4211-8874-4b5111f67fab admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:38.514444 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbeba87-249d-4211-8874-4b5111f67fab admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:38.577441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c58e9-317d-4a4a-9550-a89247aa4e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:38.578053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c58e9-317d-4a4a-9550-a89247aa4e68 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:38.578331 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c58e9-317d-4a4a-9550-a89247aa4e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:38.578604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c58e9-317d-4a4a-9550-a89247aa4e68 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:38.579292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c58e9-317d-4a4a-9550-a89247aa4e68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:38.581219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c58e9-317d-4a4a-9550-a89247aa4e68 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:38.603608 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1229/2462] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:21:38 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:21:38.608316 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3f3d2-501a-4d7e-8240-202671522d1a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:38.619157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3f3d2-501a-4d7e-8240-202671522d1a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:38.625840 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6d517-0bd2-4446-a055-933ee13fbc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:38.627553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6d517-0bd2-4446-a055-933ee13fbc8d admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:38.627553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6d517-0bd2-4446-a055-933ee13fbc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:38.627553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6d517-0bd2-4446-a055-933ee13fbc8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:38.627810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6d517-0bd2-4446-a055-933ee13fbc8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:38.629340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6d517-0bd2-4446-a055-933ee13fbc8d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:38.641588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1234/2463] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:21:38.645092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c58e9-317d-4a4a-9550-a89247aa4e68 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:38.656054 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c58e9-317d-4a4a-9550-a89247aa4e68 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:38.661017 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b658f4c-db88-4f18-ac9a-699b6421fe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:38.661642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b658f4c-db88-4f18-ac9a-699b6421fe54 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:38.661922 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b658f4c-db88-4f18-ac9a-699b6421fe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:38.662185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b658f4c-db88-4f18-ac9a-699b6421fe54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:38.662833 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b658f4c-db88-4f18-ac9a-699b6421fe54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:38.664491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b658f4c-db88-4f18-ac9a-699b6421fe54 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:38.676609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1230/2464] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:21:38.681614 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6d517-0bd2-4446-a055-933ee13fbc8d admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:38.692937 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6d517-0bd2-4446-a055-933ee13fbc8d admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:38.698849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f6153-5226-4293-9939-6c09edbb6aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:38.699612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f6153-5226-4293-9939-6c09edbb6aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:38.699877 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f6153-5226-4293-9939-6c09edbb6aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:38.700152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f6153-5226-4293-9939-6c09edbb6aaa admin admin] PATH_INFO: `/v3/projects/7db9630aac884f61a30b7b5d982d6101/users/0ac410e6e00a4694a22809cf4aa6f31d/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:38.716886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f6153-5226-4293-9939-6c09edbb6aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7db9630aac884f61a30b7b5d982d6101, user_id=0ac410e6e00a4694a22809cf4aa6f31d, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:38.719642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f6153-5226-4293-9939-6c09edbb6aaa admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:38.737277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1235/2465] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:21:38 2022] PUT /identity/v3/projects/7db9630aac884f61a30b7b5d982d6101/users/0ac410e6e00a4694a22809cf4aa6f31d/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:21:38.747714 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53168fe-b9cc-4f06-8d90-cda252b30dd5 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:38.748011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53168fe-b9cc-4f06-8d90-cda252b30dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:38.748340 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53168fe-b9cc-4f06-8d90-cda252b30dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:38.780739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-e53168fe-b9cc-4f06-8d90-cda252b30dd5 None None] MFA Rules not processed for user `0ac410e6e00a4694a22809cf4aa6f31d`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:38.782035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e53168fe-b9cc-4f06-8d90-cda252b30dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:38.918424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1231/2466] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:21:38 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:21:38.931805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f6153-5226-4293-9939-6c09edbb6aaa admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:38.933865 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-041f6153-5226-4293-9939-6c09edbb6aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:39.021745 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f6153-5226-4293-9939-6c09edbb6aaa admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:39.027967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652dc35-cb92-4bc6-ab0e-a4d0a5fa9bca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:39.028827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652dc35-cb92-4bc6-ab0e-a4d0a5fa9bca service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:39.028970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652dc35-cb92-4bc6-ab0e-a4d0a5fa9bca service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:39.029242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652dc35-cb92-4bc6-ab0e-a4d0a5fa9bca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:39.040147 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652dc35-cb92-4bc6-ab0e-a4d0a5fa9bca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:39.043093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652dc35-cb92-4bc6-ab0e-a4d0a5fa9bca service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:39.052876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1236/2467] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:38 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:48.912984 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:49.048602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=45,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:50.299812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53168fe-b9cc-4f06-8d90-cda252b30dd5 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:50.366212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53168fe-b9cc-4f06-8d90-cda252b30dd5 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:50.371418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:50.372207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:50.372483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:50.372756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] PATH_INFO: `/v3/users/0ac410e6e00a4694a22809cf4aa6f31d` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:50.373905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ac410e6e00a4694a22809cf4aa6f31d)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:50.375887 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:50.426256 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ac410e6e00a4694a22809cf4aa6f31d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:50.435430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ac410e6e00a4694a22809cf4aa6f31d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:50.436123 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ac410e6e00a4694a22809cf4aa6f31d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:50.436891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ac410e6e00a4694a22809cf4aa6f31d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:50.449968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ac410e6e00a4694a22809cf4aa6f31d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:50.462267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1232/2468] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:21:50 2022] DELETE /identity/v3/users/0ac410e6e00a4694a22809cf4aa6f31d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:21:50.467059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c071e-8431-4572-84ae-7dc23ec421a6 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:50.467587 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c071e-8431-4572-84ae-7dc23ec421a6 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:50.467723 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c071e-8431-4572-84ae-7dc23ec421a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:50.488403 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-fc9c071e-8431-4572-84ae-7dc23ec421a6 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:21:50.489635 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-fc9c071e-8431-4572-84ae-7dc23ec421a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:50.552410 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1237/2469] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:21:50.562383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:50.563831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:50.624325 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f5fe2-4729-4182-aba8-698489e2d288 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:50.628012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67c996-5ec9-4f19-829e-e6b7462ee328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:50.628626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67c996-5ec9-4f19-829e-e6b7462ee328 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:50.628908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67c996-5ec9-4f19-829e-e6b7462ee328 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:50.629173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67c996-5ec9-4f19-829e-e6b7462ee328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:50.639250 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67c996-5ec9-4f19-829e-e6b7462ee328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:50.640698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67c996-5ec9-4f19-829e-e6b7462ee328 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:50.651041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1233/2470] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:21:50.960805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c071e-8431-4572-84ae-7dc23ec421a6 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:50.962835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-fc9c071e-8431-4572-84ae-7dc23ec421a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:50.978808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c071e-8431-4572-84ae-7dc23ec421a6 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:50.984436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:50.985259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:50.985644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:50.985993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] PATH_INFO: `/v3/projects/7db9630aac884f61a30b7b5d982d6101` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:50.987280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7db9630aac884f61a30b7b5d982d6101)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:50.989367 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:51.045013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7db9630aac884f61a30b7b5d982d6101', 'request_id': 'req-42c9000e-f169-4f8b-9fff-954d144688c1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:51.051281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7db9630aac884f61a30b7b5d982d6101', 'request_id': 'req-42c9000e-f169-4f8b-9fff-954d144688c1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:51.056969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7db9630aac884f61a30b7b5d982d6101', 'request_id': 'req-42c9000e-f169-4f8b-9fff-954d144688c1'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:51.074073 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] The token cache is being invalidate because project 7db9630aac884f61a30b7b5d982d6101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:21:51.074394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:51.075694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:21:51.077350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1238/2471] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:21:50 2022] DELETE /identity/v3/projects/7db9630aac884f61a30b7b5d982d6101 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:21:55.685399 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67c996-5ec9-4f19-829e-e6b7462ee328 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:21:55.686851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ff67c996-5ec9-4f19-829e-e6b7462ee328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:55.717363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:21:55.793174 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67c996-5ec9-4f19-829e-e6b7462ee328 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:21:55.876513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbf49d-d242-4595-bbf4-56a08521aba9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:21:55.877283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbf49d-d242-4595-bbf4-56a08521aba9 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:21:55.877597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbf49d-d242-4595-bbf4-56a08521aba9 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:21:55.877902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbf49d-d242-4595-bbf4-56a08521aba9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:21:55.879977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-03cbf49d-d242-4595-bbf4-56a08521aba9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:21:55.947377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbf49d-d242-4595-bbf4-56a08521aba9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:21:55.948937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbf49d-d242-4595-bbf4-56a08521aba9 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:21:55.960015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1234/2472] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:21:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:22:01.069969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:22:05.954567 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=74 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:22:18.454444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:22:18.455924 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:18.580431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9000e-f169-4f8b-9fff-954d144688c1 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:22:18.584198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0fe05-3fdb-40bb-b752-b5f220840140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:22:18.584815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0fe05-3fdb-40bb-b752-b5f220840140 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:22:18.585093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0fe05-3fdb-40bb-b752-b5f220840140 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:22:18.585366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0fe05-3fdb-40bb-b752-b5f220840140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:22:18.587204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-78b0fe05-3fdb-40bb-b752-b5f220840140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:18.597572 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0fe05-3fdb-40bb-b752-b5f220840140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:22:18.599292 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0fe05-3fdb-40bb-b752-b5f220840140 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:22:18.607579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1239/2473] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:22:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:22:20.173717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbf49d-d242-4595-bbf4-56a08521aba9 service nova] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:22:20.175393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-03cbf49d-d242-4595-bbf4-56a08521aba9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:20.247928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbf49d-d242-4595-bbf4-56a08521aba9 service nova] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:22:20.260413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d498ed-7d92-4462-aeee-73548cee6789 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:22:20.261278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d498ed-7d92-4462-aeee-73548cee6789 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:22:20.261603 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d498ed-7d92-4462-aeee-73548cee6789 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:22:20.261856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d498ed-7d92-4462-aeee-73548cee6789 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:22:20.263706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-49d498ed-7d92-4462-aeee-73548cee6789 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:20.278185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d498ed-7d92-4462-aeee-73548cee6789 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:22:20.280129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d498ed-7d92-4462-aeee-73548cee6789 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:22:20.293761 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1235/2474] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:22:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:22:28.603869 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=48 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:22:30.289731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=28 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:22:38.315609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0fe05-3fdb-40bb-b752-b5f220840140 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:22:38.330210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0fe05-3fdb-40bb-b752-b5f220840140 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:22:38.335773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd2ebc-2935-45cc-ace9-fe77b685ff03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:22:38.336378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd2ebc-2935-45cc-ace9-fe77b685ff03 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:22:38.336665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd2ebc-2935-45cc-ace9-fe77b685ff03 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:22:38.336940 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd2ebc-2935-45cc-ace9-fe77b685ff03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:22:38.338650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3cdd2ebc-2935-45cc-ace9-fe77b685ff03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:38.398806 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd2ebc-2935-45cc-ace9-fe77b685ff03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:22:38.400485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd2ebc-2935-45cc-ace9-fe77b685ff03 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:22:38.410435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1240/2475] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:22:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:22:47.187644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d498ed-7d92-4462-aeee-73548cee6789 service nova] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:22:47.192128 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-49d498ed-7d92-4462-aeee-73548cee6789 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:47.283895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd2ebc-2935-45cc-ace9-fe77b685ff03 service nova] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:22:47.286342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3cdd2ebc-2935-45cc-ace9-fe77b685ff03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:47.330634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d498ed-7d92-4462-aeee-73548cee6789 service nova] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:22:47.337615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc471c-369d-4054-8016-4a8f407fbbe7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:22:47.338829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc471c-369d-4054-8016-4a8f407fbbe7 service placement] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:22:47.339141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc471c-369d-4054-8016-4a8f407fbbe7 service placement] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:22:47.339600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc471c-369d-4054-8016-4a8f407fbbe7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:22:47.342664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-12fc471c-369d-4054-8016-4a8f407fbbe7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:47.357185 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc471c-369d-4054-8016-4a8f407fbbe7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:22:47.358890 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc471c-369d-4054-8016-4a8f407fbbe7 service placement] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:22:47.381396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1236/2476] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:22:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:22:47.427514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd2ebc-2935-45cc-ace9-fe77b685ff03 service nova] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:22:47.432960 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7b25c-bf7f-4daa-85cf-8753559d87b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:22:47.434847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7b25c-bf7f-4daa-85cf-8753559d87b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:22:47.434847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7b25c-bf7f-4daa-85cf-8753559d87b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:22:47.435079 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7b25c-bf7f-4daa-85cf-8753559d87b1 admin admin] PATH_INFO: `/v3/users/d6083ed15f884ffea8550f28addcbfcd` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:22:47.436269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e7b25c-bf7f-4daa-85cf-8753559d87b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6083ed15f884ffea8550f28addcbfcd)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:22:47.438291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e7b25c-bf7f-4daa-85cf-8753559d87b1 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:22:47.496629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-61e7b25c-bf7f-4daa-85cf-8753559d87b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6083ed15f884ffea8550f28addcbfcd'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:22:47.514091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-61e7b25c-bf7f-4daa-85cf-8753559d87b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6083ed15f884ffea8550f28addcbfcd'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:22:47.522568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:22:47.526434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-61e7b25c-bf7f-4daa-85cf-8753559d87b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6083ed15f884ffea8550f28addcbfcd'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:22:47.528319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-61e7b25c-bf7f-4daa-85cf-8753559d87b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6083ed15f884ffea8550f28addcbfcd'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:22:47.529316 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-61e7b25c-bf7f-4daa-85cf-8753559d87b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6083ed15f884ffea8550f28addcbfcd'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:22:47.548076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1241/2477] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:22:47 2022] DELETE /identity/v3/users/d6083ed15f884ffea8550f28addcbfcd => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:22:47.765873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc471c-369d-4054-8016-4a8f407fbbe7 service placement] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:22:47.783997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc471c-369d-4054-8016-4a8f407fbbe7 service placement] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:22:47.871478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:22:47.872231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:22:47.872563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:22:47.872903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] PATH_INFO: `/v3/projects/d916ed5ea0874a0f85089a99d4a66c12` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:22:47.874099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d916ed5ea0874a0f85089a99d4a66c12)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:22:47.876140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:22:47.940165 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd916ed5ea0874a0f85089a99d4a66c12', 'request_id': 'req-89adde22-c908-47c0-9827-cccb94c42e21'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:22:47.948536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd916ed5ea0874a0f85089a99d4a66c12', 'request_id': 'req-89adde22-c908-47c0-9827-cccb94c42e21'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:22:47.955056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd916ed5ea0874a0f85089a99d4a66c12', 'request_id': 'req-89adde22-c908-47c0-9827-cccb94c42e21'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:22:47.967679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] The token cache is being invalidate because project d916ed5ea0874a0f85089a99d4a66c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:22:47.967874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:22:47.968764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:22:47.970892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1237/2478] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:22:47 2022] DELETE /identity/v3/projects/d916ed5ea0874a0f85089a99d4a66c12 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:22:49.078858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1ad2f-dae3-4692-9168-e303fa665516 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:22:49.079335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1ad2f-dae3-4692-9168-e303fa665516 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:22:49.079447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1ad2f-dae3-4692-9168-e303fa665516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:22:49.102933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-bbc1ad2f-dae3-4692-9168-e303fa665516 None None] MFA Rules not processed for user `181c888fbd454f00b049522fdfb4197e`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:22:49.103559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bbc1ad2f-dae3-4692-9168-e303fa665516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:49.159668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1242/2479] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:22:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:22:49.468564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:22:49.470214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:49.495892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:22:49.533697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adde22-c908-47c0-9827-cccb94c42e21 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:22:49.538042 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52f115-c08a-4686-87b7-cf3f2fd19c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:22:49.538644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52f115-c08a-4686-87b7-cf3f2fd19c3f service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:22:49.538919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52f115-c08a-4686-87b7-cf3f2fd19c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:22:49.539251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52f115-c08a-4686-87b7-cf3f2fd19c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:22:49.541043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1c52f115-c08a-4686-87b7-cf3f2fd19c3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:49.551215 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52f115-c08a-4686-87b7-cf3f2fd19c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:22:49.552702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52f115-c08a-4686-87b7-cf3f2fd19c3f service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:22:49.560694 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1238/2480] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:22:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:22:49.876954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1ad2f-dae3-4692-9168-e303fa665516 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:22:49.878385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bbc1ad2f-dae3-4692-9168-e303fa665516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:49.952302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1ad2f-dae3-4692-9168-e303fa665516 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:22:49.957977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eab73-742a-4041-ad6c-5923a63f184c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:22:49.958598 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eab73-742a-4041-ad6c-5923a63f184c service placement] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:22:49.958881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eab73-742a-4041-ad6c-5923a63f184c service placement] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:22:49.959181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eab73-742a-4041-ad6c-5923a63f184c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:22:49.960979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-8b7eab73-742a-4041-ad6c-5923a63f184c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:49.973321 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7eab73-742a-4041-ad6c-5923a63f184c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:22:49.974906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7eab73-742a-4041-ad6c-5923a63f184c service placement] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:22:49.985483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1243/2481] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:22:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:22:53.633468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52f115-c08a-4686-87b7-cf3f2fd19c3f service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:22:53.635090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1c52f115-c08a-4686-87b7-cf3f2fd19c3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:53.721913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52f115-c08a-4686-87b7-cf3f2fd19c3f service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:22:53.729173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbf841-f7c0-4768-bcf9-127829e44728 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:22:53.729810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbf841-f7c0-4768-bcf9-127829e44728 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:22:53.730091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbf841-f7c0-4768-bcf9-127829e44728 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:22:53.730369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbf841-f7c0-4768-bcf9-127829e44728 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:22:53.732337 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-97fbf841-f7c0-4768-bcf9-127829e44728 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:53.792926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbf841-f7c0-4768-bcf9-127829e44728 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:22:53.794899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbf841-f7c0-4768-bcf9-127829e44728 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:22:53.804784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1239/2482] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:22:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:22:59.802576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eab73-742a-4041-ad6c-5923a63f184c service placement] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:22:59.804945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-8b7eab73-742a-4041-ad6c-5923a63f184c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:59.822308 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eab73-742a-4041-ad6c-5923a63f184c service placement] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:22:59.829307 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc043f5-77d6-48d9-84f7-28702c1849fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:22:59.829946 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc043f5-77d6-48d9-84f7-28702c1849fb service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:22:59.832449 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc043f5-77d6-48d9-84f7-28702c1849fb service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:22:59.832789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc043f5-77d6-48d9-84f7-28702c1849fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:22:59.834581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-fdc043f5-77d6-48d9-84f7-28702c1849fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:22:59.848996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc043f5-77d6-48d9-84f7-28702c1849fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:22:59.850934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc043f5-77d6-48d9-84f7-28702c1849fb service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:22:59.862285 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1244/2483] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:22:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:23:03.800162 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=73 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:09.857490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=66,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:26.886552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d618e9-f2a0-4d17-8c6c-caef5585f989 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:26.887097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d618e9-f2a0-4d17-8c6c-caef5585f989 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:26.887097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d618e9-f2a0-4d17-8c6c-caef5585f989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:26.906930 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-00d618e9-f2a0-4d17-8c6c-caef5585f989 None None] MFA Rules not processed for user `406bdf61f9284ebc8620b21ee414ec91`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:23:26.908312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-00d618e9-f2a0-4d17-8c6c-caef5585f989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:26.915283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1240/2484] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:23:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:23:36.901226 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:39.955512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698b149-a65a-480d-a950-3c147549252b None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:39.956199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698b149-a65a-480d-a950-3c147549252b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:39.956199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698b149-a65a-480d-a950-3c147549252b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:39.977391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-0698b149-a65a-480d-a950-3c147549252b None None] MFA Rules not processed for user `b55eed583fac43fb9bfe1366a42bd2fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:23:39.979132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-0698b149-a65a-480d-a950-3c147549252b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:40.039333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1245/2485] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:23:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:23:50.032451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=25 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:31.474639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d618e9-f2a0-4d17-8c6c-caef5585f989 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:31.526518 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d618e9-f2a0-4d17-8c6c-caef5585f989 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:31.542988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801c0de-5ab2-4c17-9518-c9a260fd02c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:31.543679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801c0de-5ab2-4c17-9518-c9a260fd02c5 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:31.543953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801c0de-5ab2-4c17-9518-c9a260fd02c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:31.544231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801c0de-5ab2-4c17-9518-c9a260fd02c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:31.547953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b801c0de-5ab2-4c17-9518-c9a260fd02c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:31.609481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801c0de-5ab2-4c17-9518-c9a260fd02c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:31.612661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801c0de-5ab2-4c17-9518-c9a260fd02c5 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:31.641284 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1241/2486] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:33.518365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499429d-3497-4b68-8b15-27bac139cec1 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:33.518900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499429d-3497-4b68-8b15-27bac139cec1 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:33.519008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499429d-3497-4b68-8b15-27bac139cec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:33.542313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f499429d-3497-4b68-8b15-27bac139cec1 None None] MFA Rules not processed for user `57896ea1ff2046d282fe47e2a8191676`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:33.543722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f499429d-3497-4b68-8b15-27bac139cec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:33.610715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1246/2487] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:24:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:33.753323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d229b-8b2b-4784-9571-c48d60986c63 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:33.754197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d229b-8b2b-4784-9571-c48d60986c63 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:33.754651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d229b-8b2b-4784-9571-c48d60986c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:33.786164 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-5d5d229b-8b2b-4784-9571-c48d60986c63 None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:33.787436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5d5d229b-8b2b-4784-9571-c48d60986c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:33.853103 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1242/2488] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:24:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:33.941046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499429d-3497-4b68-8b15-27bac139cec1 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:33.942809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f499429d-3497-4b68-8b15-27bac139cec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:33.943288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d229b-8b2b-4784-9571-c48d60986c63 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:33.953627 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d229b-8b2b-4784-9571-c48d60986c63 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:33.958801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae32984-62e0-4f9c-b46c-0caa2e4f7704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:33.959482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae32984-62e0-4f9c-b46c-0caa2e4f7704 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:33.959829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae32984-62e0-4f9c-b46c-0caa2e4f7704 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:33.960169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae32984-62e0-4f9c-b46c-0caa2e4f7704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:33.962889 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7ae32984-62e0-4f9c-b46c-0caa2e4f7704 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:33.975080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae32984-62e0-4f9c-b46c-0caa2e4f7704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:33.977893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae32984-62e0-4f9c-b46c-0caa2e4f7704 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:33.988197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1243/2489] 10.222.0.61 () {60 vars in 1396 bytes} [Wed Sep 7 13:24:33 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:24:34.041691 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499429d-3497-4b68-8b15-27bac139cec1 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:34.046757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d98d6-9a0b-490a-b161-b6dfc0d3cb1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:34.047405 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d98d6-9a0b-490a-b161-b6dfc0d3cb1d service swift] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:34.047684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d98d6-9a0b-490a-b161-b6dfc0d3cb1d service swift] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:34.048169 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d98d6-9a0b-490a-b161-b6dfc0d3cb1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:34.057001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d98d6-9a0b-490a-b161-b6dfc0d3cb1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:34.058606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d98d6-9a0b-490a-b161-b6dfc0d3cb1d service swift] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:34.068158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1247/2490] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:24:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:24:34.072618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae32984-62e0-4f9c-b46c-0caa2e4f7704 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:34.084535 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae32984-62e0-4f9c-b46c-0caa2e4f7704 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:34.090338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd08366-cea7-4ab5-ab53-8f25ebbf5973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:34.090964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd08366-cea7-4ab5-ab53-8f25ebbf5973 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:34.091306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd08366-cea7-4ab5-ab53-8f25ebbf5973 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:34.091585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd08366-cea7-4ab5-ab53-8f25ebbf5973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:34.096529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2dd08366-cea7-4ab5-ab53-8f25ebbf5973 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:34.110339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd08366-cea7-4ab5-ab53-8f25ebbf5973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:34.111921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd08366-cea7-4ab5-ab53-8f25ebbf5973 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:34.122049 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1244/2491] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:44.063888 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=57 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:44.118245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=59 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:46.750987 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d98d6-9a0b-490a-b161-b6dfc0d3cb1d service swift] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:46.753052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f87d98d6-9a0b-490a-b161-b6dfc0d3cb1d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:46.765571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d98d6-9a0b-490a-b161-b6dfc0d3cb1d service swift] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:46.769204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd570d-4a1c-4c2b-8b9e-61b0f9c53fd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:46.769807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd570d-4a1c-4c2b-8b9e-61b0f9c53fd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:46.770088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd570d-4a1c-4c2b-8b9e-61b0f9c53fd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:46.770358 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd570d-4a1c-4c2b-8b9e-61b0f9c53fd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:46.772185 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-20dd570d-4a1c-4c2b-8b9e-61b0f9c53fd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:46.840142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd570d-4a1c-4c2b-8b9e-61b0f9c53fd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:46.841724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd570d-4a1c-4c2b-8b9e-61b0f9c53fd0 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:46.852870 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1248/2492] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:50.412063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd08366-cea7-4ab5-ab53-8f25ebbf5973 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:50.414493 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2dd08366-cea7-4ab5-ab53-8f25ebbf5973 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:50.432926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd08366-cea7-4ab5-ab53-8f25ebbf5973 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:50.438685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a26c7-94df-44a4-a042-711f6620b538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:50.439369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a26c7-94df-44a4-a042-711f6620b538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:50.439647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a26c7-94df-44a4-a042-711f6620b538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:50.439922 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a26c7-94df-44a4-a042-711f6620b538 admin admin] PATH_INFO: `/v3/users/406bdf61f9284ebc8620b21ee414ec91` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:50.441182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a26c7-94df-44a4-a042-711f6620b538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=406bdf61f9284ebc8620b21ee414ec91)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:50.443302 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a26c7-94df-44a4-a042-711f6620b538 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:50.495426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2f0a26c7-94df-44a4-a042-711f6620b538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '406bdf61f9284ebc8620b21ee414ec91'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:50.503071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2f0a26c7-94df-44a4-a042-711f6620b538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '406bdf61f9284ebc8620b21ee414ec91'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:50.503784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2f0a26c7-94df-44a4-a042-711f6620b538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '406bdf61f9284ebc8620b21ee414ec91'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:50.504426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2f0a26c7-94df-44a4-a042-711f6620b538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '406bdf61f9284ebc8620b21ee414ec91'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:50.518395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-2f0a26c7-94df-44a4-a042-711f6620b538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '406bdf61f9284ebc8620b21ee414ec91'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:50.532753 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1245/2493] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:24:50 2022] DELETE /identity/v3/users/406bdf61f9284ebc8620b21ee414ec91 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:24:50.615391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329219e-05d1-47f5-aa1c-f1878f136ea8 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:50.616091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329219e-05d1-47f5-aa1c-f1878f136ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:50.616091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329219e-05d1-47f5-aa1c-f1878f136ea8 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:50.617528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1249/2494] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:50.625174 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c27132-e6c4-4ff5-b100-58bef543a1cc None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:50.625485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c27132-e6c4-4ff5-b100-58bef543a1cc None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:50.625681 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c27132-e6c4-4ff5-b100-58bef543a1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:50.658418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-98c27132-e6c4-4ff5-b100-58bef543a1cc None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:50.659665 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-98c27132-e6c4-4ff5-b100-58bef543a1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:50.740500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329219e-05d1-47f5-aa1c-f1878f136ea8 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:50.742413 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e329219e-05d1-47f5-aa1c-f1878f136ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:50.774143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1246/2495] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:24:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:50.788973 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c27132-e6c4-4ff5-b100-58bef543a1cc None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:50.790852 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-98c27132-e6c4-4ff5-b100-58bef543a1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:50.842863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329219e-05d1-47f5-aa1c-f1878f136ea8 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:50.851564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f5c06-33ca-4d0a-867d-7856e0ecf552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:50.852464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f5c06-33ca-4d0a-867d-7856e0ecf552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:50.852664 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f5c06-33ca-4d0a-867d-7856e0ecf552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:50.853013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f5c06-33ca-4d0a-867d-7856e0ecf552 admin admin] PATH_INFO: `/v3/projects/2d2ca3f0240b478e8463f74032a1b575` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:50.854648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f5c06-33ca-4d0a-867d-7856e0ecf552 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d2ca3f0240b478e8463f74032a1b575)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:50.856688 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f5c06-33ca-4d0a-867d-7856e0ecf552 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:50.863624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c27132-e6c4-4ff5-b100-58bef543a1cc None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:50.869341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b75075-3cad-4732-afa7-6cb326470427 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f82adce75b44c6baf85b6b96dd0728', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:50.870064 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b75075-3cad-4732-afa7-6cb326470427 service glance] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:50.870340 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b75075-3cad-4732-afa7-6cb326470427 service glance] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:50.870614 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b75075-3cad-4732-afa7-6cb326470427 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:50.882435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b75075-3cad-4732-afa7-6cb326470427 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:50.884001 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b75075-3cad-4732-afa7-6cb326470427 service glance] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:50.889451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:50.898957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1247/2496] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:50.925517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-957f5c06-33ca-4d0a-867d-7856e0ecf552 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d2ca3f0240b478e8463f74032a1b575', 'request_id': 'req-957f5c06-33ca-4d0a-867d-7856e0ecf552'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:50.935523 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-957f5c06-33ca-4d0a-867d-7856e0ecf552 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d2ca3f0240b478e8463f74032a1b575', 'request_id': 'req-957f5c06-33ca-4d0a-867d-7856e0ecf552'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:50.942363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-957f5c06-33ca-4d0a-867d-7856e0ecf552 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d2ca3f0240b478e8463f74032a1b575', 'request_id': 'req-957f5c06-33ca-4d0a-867d-7856e0ecf552'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:50.960082 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-957f5c06-33ca-4d0a-867d-7856e0ecf552 admin admin] The token cache is being invalidate because project 2d2ca3f0240b478e8463f74032a1b575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:24:50.960481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-957f5c06-33ca-4d0a-867d-7856e0ecf552 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:50.961152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-957f5c06-33ca-4d0a-867d-7856e0ecf552 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:24:50.963152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1250/2497] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:24:50 2022] DELETE /identity/v3/projects/2d2ca3f0240b478e8463f74032a1b575 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:24:50.968121 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cfe14-9a55-45ee-b6c3-9626b504dcd7 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:50.968744 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cfe14-9a55-45ee-b6c3-9626b504dcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:50.969081 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cfe14-9a55-45ee-b6c3-9626b504dcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:50.980616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df910c6e-cafe-4855-a0d9-8bcc8866b145 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:50.980906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df910c6e-cafe-4855-a0d9-8bcc8866b145 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:50.981173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df910c6e-cafe-4855-a0d9-8bcc8866b145 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:50.982744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1251/2498] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:24:50.987485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408ec59-dc23-4e6c-802f-17d5ca31939f None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:50.987773 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408ec59-dc23-4e6c-802f-17d5ca31939f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:50.988050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408ec59-dc23-4e6c-802f-17d5ca31939f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:51.010982 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-1408ec59-dc23-4e6c-802f-17d5ca31939f None None] MFA Rules not processed for user `de05c08e30dd4fff93f1c97aac9553d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:51.011978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1408ec59-dc23-4e6c-802f-17d5ca31939f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:51.026064 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-160cfe14-9a55-45ee-b6c3-9626b504dcd7 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:51.027870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-160cfe14-9a55-45ee-b6c3-9626b504dcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:51.033197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:51.069766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1252/2499] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:24:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:24:51.082412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408ec59-dc23-4e6c-802f-17d5ca31939f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:51.083962 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1408ec59-dc23-4e6c-802f-17d5ca31939f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:51.093669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1248/2500] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:24:51.094353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408ec59-dc23-4e6c-802f-17d5ca31939f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:51.097995 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdcfed-597a-4313-a39d-6f5713eb1fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:51.098597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdcfed-597a-4313-a39d-6f5713eb1fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:51.098871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdcfed-597a-4313-a39d-6f5713eb1fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:51.099165 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdcfed-597a-4313-a39d-6f5713eb1fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:51.102543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dce7d-f0ed-4f28-8055-10a61a84e7da None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:51.102831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dce7d-f0ed-4f28-8055-10a61a84e7da None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:51.103122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dce7d-f0ed-4f28-8055-10a61a84e7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:51.107418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdcfed-597a-4313-a39d-6f5713eb1fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:51.108844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdcfed-597a-4313-a39d-6f5713eb1fb9 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:51.116760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1253/2501] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:24:51.124077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-749dce7d-f0ed-4f28-8055-10a61a84e7da None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:51.125326 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-749dce7d-f0ed-4f28-8055-10a61a84e7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:51.133307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1249/2502] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:24:51.137552 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdcfed-597a-4313-a39d-6f5713eb1fb9 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:51.149323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdcfed-597a-4313-a39d-6f5713eb1fb9 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:51.154009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf79d4c-70c1-4f2b-ae7b-77d1dbcdc6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:51.154611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf79d4c-70c1-4f2b-ae7b-77d1dbcdc6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:51.154897 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf79d4c-70c1-4f2b-ae7b-77d1dbcdc6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:51.155273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf79d4c-70c1-4f2b-ae7b-77d1dbcdc6f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:51.155927 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf79d4c-70c1-4f2b-ae7b-77d1dbcdc6f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:24:51.156232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf79d4c-70c1-4f2b-ae7b-77d1dbcdc6f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:51.157823 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf79d4c-70c1-4f2b-ae7b-77d1dbcdc6f4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:51.173276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1254/2503] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:24:51.177642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dce7d-f0ed-4f28-8055-10a61a84e7da None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:51.188118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dce7d-f0ed-4f28-8055-10a61a84e7da None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:51.192945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9172d-07ea-4aca-ba82-b127db06a641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:51.193559 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9172d-07ea-4aca-ba82-b127db06a641 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:51.193835 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9172d-07ea-4aca-ba82-b127db06a641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:51.194104 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9172d-07ea-4aca-ba82-b127db06a641 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:51.194780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9172d-07ea-4aca-ba82-b127db06a641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:51.196738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9172d-07ea-4aca-ba82-b127db06a641 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:51.236328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1250/2504] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:24:51 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:24:51.240501 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf79d4c-70c1-4f2b-ae7b-77d1dbcdc6f4 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:51.251410 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf79d4c-70c1-4f2b-ae7b-77d1dbcdc6f4 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:51.312044 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487c807-6e7f-4948-9a80-5721ecb4c383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:51.312671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487c807-6e7f-4948-9a80-5721ecb4c383 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:51.312927 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487c807-6e7f-4948-9a80-5721ecb4c383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:51.313193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487c807-6e7f-4948-9a80-5721ecb4c383 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:51.313843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487c807-6e7f-4948-9a80-5721ecb4c383 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:51.315861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487c807-6e7f-4948-9a80-5721ecb4c383 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:51.343144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1255/2505] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:24:51 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:24:51.347661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9172d-07ea-4aca-ba82-b127db06a641 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:51.361570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9172d-07ea-4aca-ba82-b127db06a641 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:51.366692 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefeb639-43a5-4466-8f89-f374e9e8f420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:51.367426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb639-43a5-4466-8f89-f374e9e8f420 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:51.367776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb639-43a5-4466-8f89-f374e9e8f420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:51.368124 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb639-43a5-4466-8f89-f374e9e8f420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:51.368885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefeb639-43a5-4466-8f89-f374e9e8f420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:51.370779 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefeb639-43a5-4466-8f89-f374e9e8f420 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:51.383093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1251/2506] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:24:51.389403 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487c807-6e7f-4948-9a80-5721ecb4c383 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:51.401678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487c807-6e7f-4948-9a80-5721ecb4c383 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:51.408917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d644f-506f-4b46-be02-27f5aa7e2db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:51.410021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d644f-506f-4b46-be02-27f5aa7e2db1 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:51.410547 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d644f-506f-4b46-be02-27f5aa7e2db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:51.410919 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d644f-506f-4b46-be02-27f5aa7e2db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:51.411670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d644f-506f-4b46-be02-27f5aa7e2db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:51.413297 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d644f-506f-4b46-be02-27f5aa7e2db1 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:51.427145 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1256/2507] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:24:51.430047 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefeb639-43a5-4466-8f89-f374e9e8f420 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:51.441563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefeb639-43a5-4466-8f89-f374e9e8f420 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:51.446594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b5653-1216-41a8-b5b2-50224ffc6ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:51.447354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b5653-1216-41a8-b5b2-50224ffc6ced admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:51.447700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b5653-1216-41a8-b5b2-50224ffc6ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:51.448038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b5653-1216-41a8-b5b2-50224ffc6ced admin admin] PATH_INFO: `/v3/projects/5508857410f645289349edbaae208dc4/users/98ac40f3a3f941419749926e48b9fb1b/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:51.466546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b5653-1216-41a8-b5b2-50224ffc6ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=5508857410f645289349edbaae208dc4, user_id=98ac40f3a3f941419749926e48b9fb1b, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:51.469880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b5653-1216-41a8-b5b2-50224ffc6ced admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:51.487915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1252/2508] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:24:51 2022] PUT /identity/v3/projects/5508857410f645289349edbaae208dc4/users/98ac40f3a3f941419749926e48b9fb1b/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:24:51.499149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b76670-5766-4369-8af6-50159ee1c171 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:51.499605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b76670-5766-4369-8af6-50159ee1c171 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:51.499998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b76670-5766-4369-8af6-50159ee1c171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:51.543863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-85b76670-5766-4369-8af6-50159ee1c171 None None] MFA Rules not processed for user `98ac40f3a3f941419749926e48b9fb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:51.551676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-85b76670-5766-4369-8af6-50159ee1c171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:51.670152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1257/2509] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:24:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:51.683050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b5653-1216-41a8-b5b2-50224ffc6ced admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:51.686368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-000b5653-1216-41a8-b5b2-50224ffc6ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:51.787536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:51.797536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b5653-1216-41a8-b5b2-50224ffc6ced admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:51.801561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a05ea7-6c2a-4a4d-862b-273aeb1c9c94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:51.802013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a05ea7-6c2a-4a4d-862b-273aeb1c9c94 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:51.802292 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a05ea7-6c2a-4a4d-862b-273aeb1c9c94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:51.802569 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a05ea7-6c2a-4a4d-862b-273aeb1c9c94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:51.814578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a05ea7-6c2a-4a4d-862b-273aeb1c9c94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:51.816362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a05ea7-6c2a-4a4d-862b-273aeb1c9c94 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:51.828781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1253/2510] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:53.665431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550b639-b803-478a-ad85-f37d85f87082 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:53.665927 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550b639-b803-478a-ad85-f37d85f87082 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:53.666149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550b639-b803-478a-ad85-f37d85f87082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:53.687391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-4550b639-b803-478a-ad85-f37d85f87082 None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:53.688646 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4550b639-b803-478a-ad85-f37d85f87082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:53.699503 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:53.750681 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1258/2511] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:24:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:53.763433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a05ea7-6c2a-4a4d-862b-273aeb1c9c94 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:53.764846 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-46a05ea7-6c2a-4a4d-862b-273aeb1c9c94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:53.835563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a05ea7-6c2a-4a4d-862b-273aeb1c9c94 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:53.840654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa1e2f-f9a9-4523-a2fb-9a9317be7356 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:53.841346 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa1e2f-f9a9-4523-a2fb-9a9317be7356 service swift] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:53.841623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa1e2f-f9a9-4523-a2fb-9a9317be7356 service swift] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:53.841893 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa1e2f-f9a9-4523-a2fb-9a9317be7356 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:53.850465 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa1e2f-f9a9-4523-a2fb-9a9317be7356 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:53.852655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa1e2f-f9a9-4523-a2fb-9a9317be7356 service swift] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:53.861360 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1254/2512] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:24:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:25:00.761650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550b639-b803-478a-ad85-f37d85f87082 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:00.848837 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550b639-b803-478a-ad85-f37d85f87082 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:00.856208 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:00.857172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:00.857763 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:00.858188 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] PATH_INFO: `/v3/users/98ac40f3a3f941419749926e48b9fb1b` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:00.859891 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ac40f3a3f941419749926e48b9fb1b)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:00.862370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:00.935606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ac40f3a3f941419749926e48b9fb1b'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:00.966012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ac40f3a3f941419749926e48b9fb1b'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:00.984461 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ac40f3a3f941419749926e48b9fb1b'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:00.984709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ac40f3a3f941419749926e48b9fb1b'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:00.985528 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ac40f3a3f941419749926e48b9fb1b'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:01.005042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1259/2513] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:25:00 2022] DELETE /identity/v3/users/98ac40f3a3f941419749926e48b9fb1b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:01.009911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a73c45-59c2-49e7-80ec-32827b49649e None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:01.010451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a73c45-59c2-49e7-80ec-32827b49649e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:01.010602 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a73c45-59c2-49e7-80ec-32827b49649e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:01.030600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-13a73c45-59c2-49e7-80ec-32827b49649e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:01.031838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-13a73c45-59c2-49e7-80ec-32827b49649e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:01.143125 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:01.145322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:01.147268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1255/2514] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:01.191941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:01.229799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652df40-ea34-4a9d-81ff-9e0e05077669 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:01.233958 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc80d94-d6b4-4a4a-944e-733ddf888c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:01.234932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc80d94-d6b4-4a4a-944e-733ddf888c82 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:01.235745 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc80d94-d6b4-4a4a-944e-733ddf888c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:01.236034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc80d94-d6b4-4a4a-944e-733ddf888c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:01.247042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc80d94-d6b4-4a4a-944e-733ddf888c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:01.248556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc80d94-d6b4-4a4a-944e-733ddf888c82 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:01.260281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1260/2515] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:25:01.605383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a73c45-59c2-49e7-80ec-32827b49649e None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:01.606871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-13a73c45-59c2-49e7-80ec-32827b49649e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:01.621189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a73c45-59c2-49e7-80ec-32827b49649e None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:01.626535 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e027a-d181-410e-b795-82667d4d3512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:01.627291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e027a-d181-410e-b795-82667d4d3512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:01.627574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e027a-d181-410e-b795-82667d4d3512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:01.627849 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e027a-d181-410e-b795-82667d4d3512 admin admin] PATH_INFO: `/v3/projects/5508857410f645289349edbaae208dc4` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:01.629697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e027a-d181-410e-b795-82667d4d3512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5508857410f645289349edbaae208dc4)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:01.631866 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e027a-d181-410e-b795-82667d4d3512 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:01.697882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c98e027a-d181-410e-b795-82667d4d3512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5508857410f645289349edbaae208dc4', 'request_id': 'req-c98e027a-d181-410e-b795-82667d4d3512'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:01.724679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c98e027a-d181-410e-b795-82667d4d3512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5508857410f645289349edbaae208dc4', 'request_id': 'req-c98e027a-d181-410e-b795-82667d4d3512'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:01.731938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c98e027a-d181-410e-b795-82667d4d3512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5508857410f645289349edbaae208dc4', 'request_id': 'req-c98e027a-d181-410e-b795-82667d4d3512'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:01.751731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:01.757088 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c98e027a-d181-410e-b795-82667d4d3512 admin admin] The token cache is being invalidate because project 5508857410f645289349edbaae208dc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:25:01.758160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c98e027a-d181-410e-b795-82667d4d3512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:01.759251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c98e027a-d181-410e-b795-82667d4d3512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:01.763363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1256/2516] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:25:01 2022] DELETE /identity/v3/projects/5508857410f645289349edbaae208dc4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:01.766376 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cc085-58ff-4755-b601-610d6c82f469 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:01.766945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cc085-58ff-4755-b601-610d6c82f469 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:01.768141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cc085-58ff-4755-b601-610d6c82f469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:01.797573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-7d9cc085-58ff-4755-b601-610d6c82f469 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:01.799249 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7d9cc085-58ff-4755-b601-610d6c82f469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:01.884081 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1261/2517] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:25:01.892052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb01d6-84ea-4403-b2cf-205853b54238 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:01.892213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb01d6-84ea-4403-b2cf-205853b54238 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:01.892433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb01d6-84ea-4403-b2cf-205853b54238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:01.925915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-7acb01d6-84ea-4403-b2cf-205853b54238 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:01.927213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7acb01d6-84ea-4403-b2cf-205853b54238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:01.940539 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1257/2518] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:01.942180 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cc085-58ff-4755-b601-610d6c82f469 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:01.953218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cc085-58ff-4755-b601-610d6c82f469 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:01.957782 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd178291-3dd5-45f7-9568-0ffa444ce493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:01.958927 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd178291-3dd5-45f7-9568-0ffa444ce493 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:01.959257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd178291-3dd5-45f7-9568-0ffa444ce493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:01.959533 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd178291-3dd5-45f7-9568-0ffa444ce493 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:01.960210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd178291-3dd5-45f7-9568-0ffa444ce493 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:25:01.960523 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd178291-3dd5-45f7-9568-0ffa444ce493 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:01.962148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd178291-3dd5-45f7-9568-0ffa444ce493 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:01.993956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1262/2519] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:25:01.996535 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb01d6-84ea-4403-b2cf-205853b54238 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.006789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb01d6-84ea-4403-b2cf-205853b54238 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.013135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d39af9-7f9a-4b5e-81b7-9110b4983abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.014018 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39af9-7f9a-4b5e-81b7-9110b4983abf admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.014253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39af9-7f9a-4b5e-81b7-9110b4983abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.014516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39af9-7f9a-4b5e-81b7-9110b4983abf admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.015235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d39af9-7f9a-4b5e-81b7-9110b4983abf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:02.017397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d39af9-7f9a-4b5e-81b7-9110b4983abf admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:02.054615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1258/2520] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:25:01 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:02.058876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd178291-3dd5-45f7-9568-0ffa444ce493 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.070524 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd178291-3dd5-45f7-9568-0ffa444ce493 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.152969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdf781-f776-4383-91e2-9b227f4d5cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.154027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdf781-f776-4383-91e2-9b227f4d5cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.154306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdf781-f776-4383-91e2-9b227f4d5cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.154578 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdf781-f776-4383-91e2-9b227f4d5cc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.155274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdf781-f776-4383-91e2-9b227f4d5cc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:02.157224 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdf781-f776-4383-91e2-9b227f4d5cc0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:02.182802 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1263/2521] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:25:02 2022] POST /identity/v3/users => generated 283 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:25:02.188642 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d39af9-7f9a-4b5e-81b7-9110b4983abf admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.200873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d39af9-7f9a-4b5e-81b7-9110b4983abf admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.214603 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc9493-4a38-4592-b649-4671b6112b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.216048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc9493-4a38-4592-b649-4671b6112b30 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.216334 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc9493-4a38-4592-b649-4671b6112b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.216608 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc9493-4a38-4592-b649-4671b6112b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.217271 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc9493-4a38-4592-b649-4671b6112b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:02.219025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc9493-4a38-4592-b649-4671b6112b30 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:02.232476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1259/2522] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:02.236731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdf781-f776-4383-91e2-9b227f4d5cc0 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.247949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdf781-f776-4383-91e2-9b227f4d5cc0 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.253792 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a97629-25e5-4259-a347-6f95bf25ea42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.254993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a97629-25e5-4259-a347-6f95bf25ea42 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.254993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a97629-25e5-4259-a347-6f95bf25ea42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.254993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a97629-25e5-4259-a347-6f95bf25ea42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.255619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a97629-25e5-4259-a347-6f95bf25ea42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:02.257441 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a97629-25e5-4259-a347-6f95bf25ea42 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:02.269356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1264/2523] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:02.273609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc9493-4a38-4592-b649-4671b6112b30 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.283813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc9493-4a38-4592-b649-4671b6112b30 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.289431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5df1c-d5c1-4cab-8c23-5886b5daf6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.290084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5df1c-d5c1-4cab-8c23-5886b5daf6df admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.290362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5df1c-d5c1-4cab-8c23-5886b5daf6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.290632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5df1c-d5c1-4cab-8c23-5886b5daf6df admin admin] PATH_INFO: `/v3/projects/90227e45126e4887bf4a4998b910ab85/users/64c55c8f22cc44e9a6acc4046aa544f4/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.335497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5df1c-d5c1-4cab-8c23-5886b5daf6df admin admin] RBAC: Authorizing `identity:create_grant(project_id=90227e45126e4887bf4a4998b910ab85, user_id=64c55c8f22cc44e9a6acc4046aa544f4, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:02.339236 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5df1c-d5c1-4cab-8c23-5886b5daf6df admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:02.354878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1260/2524] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:25:02 2022] PUT /identity/v3/projects/90227e45126e4887bf4a4998b910ab85/users/64c55c8f22cc44e9a6acc4046aa544f4/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:02.365046 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15440e0f-8746-4720-84f2-c5b8baac475d None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.365360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15440e0f-8746-4720-84f2-c5b8baac475d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.365634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15440e0f-8746-4720-84f2-c5b8baac475d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.395076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-15440e0f-8746-4720-84f2-c5b8baac475d None None] MFA Rules not processed for user `64c55c8f22cc44e9a6acc4046aa544f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:02.396302 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-15440e0f-8746-4720-84f2-c5b8baac475d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:02.435957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:02.468264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1265/2525] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:02.472689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5df1c-d5c1-4cab-8c23-5886b5daf6df admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.536849 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5df1c-d5c1-4cab-8c23-5886b5daf6df admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.542283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4031e9-b50b-4d36-9844-f666e0e9ba80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.542954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4031e9-b50b-4d36-9844-f666e0e9ba80 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.543223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4031e9-b50b-4d36-9844-f666e0e9ba80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.543501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4031e9-b50b-4d36-9844-f666e0e9ba80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.544155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4031e9-b50b-4d36-9844-f666e0e9ba80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:02.546088 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4031e9-b50b-4d36-9844-f666e0e9ba80 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:02.577984 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1261/2526] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:25:02 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:02.579140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15440e0f-8746-4720-84f2-c5b8baac475d None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.593630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15440e0f-8746-4720-84f2-c5b8baac475d None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.658734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4bcc4-353b-4f5d-b8f8-3bd6b91bc288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.659378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4bcc4-353b-4f5d-b8f8-3bd6b91bc288 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.659820 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4bcc4-353b-4f5d-b8f8-3bd6b91bc288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.660295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4bcc4-353b-4f5d-b8f8-3bd6b91bc288 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.660950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4bcc4-353b-4f5d-b8f8-3bd6b91bc288 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:02.662864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4bcc4-353b-4f5d-b8f8-3bd6b91bc288 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:02.687152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1266/2527] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:25:02 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:02.692550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4031e9-b50b-4d36-9844-f666e0e9ba80 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.703934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4031e9-b50b-4d36-9844-f666e0e9ba80 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.708940 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2aedf4-0b1c-40c1-91a6-0771a3d3afd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.709553 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aedf4-0b1c-40c1-91a6-0771a3d3afd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.709828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aedf4-0b1c-40c1-91a6-0771a3d3afd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.710098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aedf4-0b1c-40c1-91a6-0771a3d3afd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.711360 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2aedf4-0b1c-40c1-91a6-0771a3d3afd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:02.712983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2aedf4-0b1c-40c1-91a6-0771a3d3afd3 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:02.725243 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1262/2528] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:02.730580 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4bcc4-353b-4f5d-b8f8-3bd6b91bc288 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.743806 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4bcc4-353b-4f5d-b8f8-3bd6b91bc288 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.749451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e8121-d325-498f-b0ba-437b360aa840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.750239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e8121-d325-498f-b0ba-437b360aa840 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.750687 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e8121-d325-498f-b0ba-437b360aa840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.751034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e8121-d325-498f-b0ba-437b360aa840 admin admin] PATH_INFO: `/v3/projects/c024b6c288534cb1b19fb9b34a9f8c95/users/a591d7b8ae5e4628bf068ab1bed94fe8/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.767573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e8121-d325-498f-b0ba-437b360aa840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c024b6c288534cb1b19fb9b34a9f8c95, user_id=a591d7b8ae5e4628bf068ab1bed94fe8, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:02.770265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e8121-d325-498f-b0ba-437b360aa840 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:02.790075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1267/2529] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:25:02 2022] PUT /identity/v3/projects/c024b6c288534cb1b19fb9b34a9f8c95/users/a591d7b8ae5e4628bf068ab1bed94fe8/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:02.795003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2aedf4-0b1c-40c1-91a6-0771a3d3afd3 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.811686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2aedf4-0b1c-40c1-91a6-0771a3d3afd3 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.876133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7e989-7106-4ed9-94a3-470b1e4238ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.876776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7e989-7106-4ed9-94a3-470b1e4238ac admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.877045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7e989-7106-4ed9-94a3-470b1e4238ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.877318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7e989-7106-4ed9-94a3-470b1e4238ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.877970 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7e989-7106-4ed9-94a3-470b1e4238ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:02.879658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7e989-7106-4ed9-94a3-470b1e4238ac admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:02.890658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1263/2530] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:02.895023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e8121-d325-498f-b0ba-437b360aa840 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.906491 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e8121-d325-498f-b0ba-437b360aa840 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.911373 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75c002-facb-4bae-9987-a8fb2d1296ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.912036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75c002-facb-4bae-9987-a8fb2d1296ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.912346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75c002-facb-4bae-9987-a8fb2d1296ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.912676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75c002-facb-4bae-9987-a8fb2d1296ec admin admin] PATH_INFO: `/v3/projects/c024b6c288534cb1b19fb9b34a9f8c95/users/a591d7b8ae5e4628bf068ab1bed94fe8/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.914566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75c002-facb-4bae-9987-a8fb2d1296ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c024b6c288534cb1b19fb9b34a9f8c95, user_id=a591d7b8ae5e4628bf068ab1bed94fe8, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:02.918239 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75c002-facb-4bae-9987-a8fb2d1296ec admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:02.932162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1268/2531] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:25:02 2022] PUT /identity/v3/projects/c024b6c288534cb1b19fb9b34a9f8c95/users/a591d7b8ae5e4628bf068ab1bed94fe8/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:02.942682 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d210a-4824-4e48-a906-19e8341be7cc None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.942885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d210a-4824-4e48-a906-19e8341be7cc None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.943190 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d210a-4824-4e48-a906-19e8341be7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.958197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:02.975736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-056d210a-4824-4e48-a906-19e8341be7cc None None] MFA Rules not processed for user `a591d7b8ae5e4628bf068ab1bed94fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:02.977098 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-056d210a-4824-4e48-a906-19e8341be7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:03.051188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1264/2532] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:03.063148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75c002-facb-4bae-9987-a8fb2d1296ec admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:03.065158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ce75c002-facb-4bae-9987-a8fb2d1296ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:03.141333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75c002-facb-4bae-9987-a8fb2d1296ec admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:03.144970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9563a7f-ca41-4ce0-93de-9be74c3256f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:03.145576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9563a7f-ca41-4ce0-93de-9be74c3256f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:03.145859 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9563a7f-ca41-4ce0-93de-9be74c3256f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:03.146132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9563a7f-ca41-4ce0-93de-9be74c3256f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:03.155737 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9563a7f-ca41-4ce0-93de-9be74c3256f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:03.157160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9563a7f-ca41-4ce0-93de-9be74c3256f9 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:03.166946 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1269/2533] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:03.614729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d210a-4824-4e48-a906-19e8341be7cc None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:03.681562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d210a-4824-4e48-a906-19e8341be7cc None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:03.686871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:03.687581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:03.687863 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:03.688135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] PATH_INFO: `/v3/users/64c55c8f22cc44e9a6acc4046aa544f4` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:03.689274 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c55c8f22cc44e9a6acc4046aa544f4)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:03.691313 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:03.737902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c55c8f22cc44e9a6acc4046aa544f4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:03.749080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c55c8f22cc44e9a6acc4046aa544f4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:03.749080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c55c8f22cc44e9a6acc4046aa544f4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:03.749080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c55c8f22cc44e9a6acc4046aa544f4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:03.764172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c55c8f22cc44e9a6acc4046aa544f4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:03.776463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1265/2534] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:25:03 2022] DELETE /identity/v3/users/64c55c8f22cc44e9a6acc4046aa544f4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:03.781115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf5edc-ae03-4169-88e6-93815995382a None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:03.781619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf5edc-ae03-4169-88e6-93815995382a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:03.781716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf5edc-ae03-4169-88e6-93815995382a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:03.801039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-ddaf5edc-ae03-4169-88e6-93815995382a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:03.802256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ddaf5edc-ae03-4169-88e6-93815995382a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:03.832721 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:03.866437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1270/2535] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:03.877051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:03.878466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:03.892206 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:03.942295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27777e28-5e0a-4877-ab9c-1aa472a06a45 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:03.946545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef0f55-ef36-4d21-aeee-768ff97fd379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:03.947183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef0f55-ef36-4d21-aeee-768ff97fd379 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:03.947463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef0f55-ef36-4d21-aeee-768ff97fd379 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:03.947725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef0f55-ef36-4d21-aeee-768ff97fd379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:03.957268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef0f55-ef36-4d21-aeee-768ff97fd379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:03.958705 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef0f55-ef36-4d21-aeee-768ff97fd379 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:03.968652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1266/2536] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:04.320033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf5edc-ae03-4169-88e6-93815995382a None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:04.321534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ddaf5edc-ae03-4169-88e6-93815995382a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:04.346959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf5edc-ae03-4169-88e6-93815995382a None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:04.352198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:04.352832 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:04.353111 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:04.353382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] PATH_INFO: `/v3/projects/90227e45126e4887bf4a4998b910ab85` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:04.354746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90227e45126e4887bf4a4998b910ab85)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:04.356717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:04.405023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90227e45126e4887bf4a4998b910ab85', 'request_id': 'req-0b58656f-a364-44f4-8103-870ef848b7e5'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:04.411261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90227e45126e4887bf4a4998b910ab85', 'request_id': 'req-0b58656f-a364-44f4-8103-870ef848b7e5'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:04.417021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90227e45126e4887bf4a4998b910ab85', 'request_id': 'req-0b58656f-a364-44f4-8103-870ef848b7e5'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:04.430746 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] The token cache is being invalidate because project 90227e45126e4887bf4a4998b910ab85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:25:04.431341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:04.432086 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:04.433780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1271/2537] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:25:04 2022] DELETE /identity/v3/projects/90227e45126e4887bf4a4998b910ab85 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:04.438436 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef0f55-ef36-4d21-aeee-768ff97fd379 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:04.451540 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef0f55-ef36-4d21-aeee-768ff97fd379 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:04.515638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86784e-2ad1-4d0c-9ecf-4a80c289bd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:04.516372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86784e-2ad1-4d0c-9ecf-4a80c289bd95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:04.516649 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86784e-2ad1-4d0c-9ecf-4a80c289bd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:04.516919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86784e-2ad1-4d0c-9ecf-4a80c289bd95 admin admin] PATH_INFO: `/v3/users/a591d7b8ae5e4628bf068ab1bed94fe8` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:04.518230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86784e-2ad1-4d0c-9ecf-4a80c289bd95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a591d7b8ae5e4628bf068ab1bed94fe8)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:04.520565 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86784e-2ad1-4d0c-9ecf-4a80c289bd95 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:04.572491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1e86784e-2ad1-4d0c-9ecf-4a80c289bd95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a591d7b8ae5e4628bf068ab1bed94fe8'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:04.581728 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1e86784e-2ad1-4d0c-9ecf-4a80c289bd95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a591d7b8ae5e4628bf068ab1bed94fe8'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:04.582347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1e86784e-2ad1-4d0c-9ecf-4a80c289bd95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a591d7b8ae5e4628bf068ab1bed94fe8'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:04.583060 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1e86784e-2ad1-4d0c-9ecf-4a80c289bd95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a591d7b8ae5e4628bf068ab1bed94fe8'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:04.597127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1e86784e-2ad1-4d0c-9ecf-4a80c289bd95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a591d7b8ae5e4628bf068ab1bed94fe8'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:04.610675 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1267/2538] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:25:04 2022] DELETE /identity/v3/users/a591d7b8ae5e4628bf068ab1bed94fe8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:04.917559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:04.919068 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:04.935955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58656f-a364-44f4-8103-870ef848b7e5 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:04.998788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4faee9-72e3-4b14-a27e-162ae1094040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:04.999471 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4faee9-72e3-4b14-a27e-162ae1094040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:04.999756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4faee9-72e3-4b14-a27e-162ae1094040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.000065 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4faee9-72e3-4b14-a27e-162ae1094040 admin admin] PATH_INFO: `/v3/projects/c024b6c288534cb1b19fb9b34a9f8c95` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.001217 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4faee9-72e3-4b14-a27e-162ae1094040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c024b6c288534cb1b19fb9b34a9f8c95)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:05.003162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4faee9-72e3-4b14-a27e-162ae1094040 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:05.058886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-7d4faee9-72e3-4b14-a27e-162ae1094040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c024b6c288534cb1b19fb9b34a9f8c95', 'request_id': 'req-7d4faee9-72e3-4b14-a27e-162ae1094040'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:05.065284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-7d4faee9-72e3-4b14-a27e-162ae1094040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c024b6c288534cb1b19fb9b34a9f8c95', 'request_id': 'req-7d4faee9-72e3-4b14-a27e-162ae1094040'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:05.069217 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:05.071624 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-7d4faee9-72e3-4b14-a27e-162ae1094040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c024b6c288534cb1b19fb9b34a9f8c95', 'request_id': 'req-7d4faee9-72e3-4b14-a27e-162ae1094040'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:05.091817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-7d4faee9-72e3-4b14-a27e-162ae1094040 admin admin] The token cache is being invalidate because project c024b6c288534cb1b19fb9b34a9f8c95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:25:05.092023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-7d4faee9-72e3-4b14-a27e-162ae1094040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:05.092658 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-7d4faee9-72e3-4b14-a27e-162ae1094040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:05.094211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1272/2539] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:25:04 2022] DELETE /identity/v3/projects/c024b6c288534cb1b19fb9b34a9f8c95 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:05.099075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcee80e-9438-43e9-a79f-395e6f1f93e8 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.099385 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcee80e-9438-43e9-a79f-395e6f1f93e8 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.099655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcee80e-9438-43e9-a79f-395e6f1f93e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.121297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-afcee80e-9438-43e9-a79f-395e6f1f93e8 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:05.122491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-afcee80e-9438-43e9-a79f-395e6f1f93e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:05.151463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:05.183896 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1268/2540] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:05.192875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f54396-661a-471f-aab0-6d865bfddab5 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.193169 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f54396-661a-471f-aab0-6d865bfddab5 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.193447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f54396-661a-471f-aab0-6d865bfddab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.213916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a6f54396-661a-471f-aab0-6d865bfddab5 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:05.215381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a6f54396-661a-471f-aab0-6d865bfddab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:05.223540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1273/2541] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:05.228342 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcee80e-9438-43e9-a79f-395e6f1f93e8 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:05.239670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcee80e-9438-43e9-a79f-395e6f1f93e8 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:05.246075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88178af-9d93-48b5-b795-05d4f00d5b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:05.246700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88178af-9d93-48b5-b795-05d4f00d5b67 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.246970 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88178af-9d93-48b5-b795-05d4f00d5b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.247275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88178af-9d93-48b5-b795-05d4f00d5b67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.247933 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88178af-9d93-48b5-b795-05d4f00d5b67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:25:05.248236 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88178af-9d93-48b5-b795-05d4f00d5b67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:05.249890 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88178af-9d93-48b5-b795-05d4f00d5b67 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:05.263587 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1269/2542] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:25:05.268039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f54396-661a-471f-aab0-6d865bfddab5 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:05.278267 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f54396-661a-471f-aab0-6d865bfddab5 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:05.283085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac828f-66fa-4934-9c12-c47196d864c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:05.283712 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac828f-66fa-4934-9c12-c47196d864c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.283985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac828f-66fa-4934-9c12-c47196d864c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.284281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac828f-66fa-4934-9c12-c47196d864c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.284918 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac828f-66fa-4934-9c12-c47196d864c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:05.286796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac828f-66fa-4934-9c12-c47196d864c0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:05.321379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1274/2543] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:25:05 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:05.323618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88178af-9d93-48b5-b795-05d4f00d5b67 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:05.333918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88178af-9d93-48b5-b795-05d4f00d5b67 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:05.396511 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385af8ae-326b-41e2-8beb-380994df6d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:05.397130 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385af8ae-326b-41e2-8beb-380994df6d9e admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.397411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385af8ae-326b-41e2-8beb-380994df6d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.397684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385af8ae-326b-41e2-8beb-380994df6d9e admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.398901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385af8ae-326b-41e2-8beb-380994df6d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:05.401090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385af8ae-326b-41e2-8beb-380994df6d9e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:05.423627 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1270/2544] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:25:05 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:05.427728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac828f-66fa-4934-9c12-c47196d864c0 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:05.438583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac828f-66fa-4934-9c12-c47196d864c0 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:05.443344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03685b1-aa73-48fe-8140-e62da879046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:05.443956 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03685b1-aa73-48fe-8140-e62da879046b admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.444240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03685b1-aa73-48fe-8140-e62da879046b admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.444516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03685b1-aa73-48fe-8140-e62da879046b admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.445199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03685b1-aa73-48fe-8140-e62da879046b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:05.446803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03685b1-aa73-48fe-8140-e62da879046b admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:05.457608 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1275/2545] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:05.462116 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385af8ae-326b-41e2-8beb-380994df6d9e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:05.472397 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385af8ae-326b-41e2-8beb-380994df6d9e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:05.477900 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f86ee2-6b13-48cb-8ae1-fe40c75a6265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:05.478646 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86ee2-6b13-48cb-8ae1-fe40c75a6265 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.478864 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86ee2-6b13-48cb-8ae1-fe40c75a6265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.479549 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86ee2-6b13-48cb-8ae1-fe40c75a6265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.480793 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f86ee2-6b13-48cb-8ae1-fe40c75a6265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:05.482235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f86ee2-6b13-48cb-8ae1-fe40c75a6265 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:05.494229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1271/2546] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:05.498193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03685b1-aa73-48fe-8140-e62da879046b admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:05.508374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03685b1-aa73-48fe-8140-e62da879046b admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:05.518368 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acd713-a351-44c8-a54c-f33dc7b90222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:05.519291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acd713-a351-44c8-a54c-f33dc7b90222 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.519574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acd713-a351-44c8-a54c-f33dc7b90222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.519850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acd713-a351-44c8-a54c-f33dc7b90222 admin admin] PATH_INFO: `/v3/projects/cf35a719d9774906a8a073362cfb4d88/users/2ab6c4f021594060bb18d0ba9692d1a6/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.539593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42acd713-a351-44c8-a54c-f33dc7b90222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf35a719d9774906a8a073362cfb4d88, user_id=2ab6c4f021594060bb18d0ba9692d1a6, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:05.542253 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42acd713-a351-44c8-a54c-f33dc7b90222 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:05.556933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1276/2547] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:25:05 2022] PUT /identity/v3/projects/cf35a719d9774906a8a073362cfb4d88/users/2ab6c4f021594060bb18d0ba9692d1a6/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:05.567391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4fdf6-dd05-4a3e-9499-a124a430420f None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.567679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4fdf6-dd05-4a3e-9499-a124a430420f None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.567956 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4fdf6-dd05-4a3e-9499-a124a430420f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.596194 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8ec4fdf6-dd05-4a3e-9499-a124a430420f None None] MFA Rules not processed for user `2ab6c4f021594060bb18d0ba9692d1a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:05.597408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8ec4fdf6-dd05-4a3e-9499-a124a430420f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:05.666946 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1272/2548] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:05.671432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acd713-a351-44c8-a54c-f33dc7b90222 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:05.736677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acd713-a351-44c8-a54c-f33dc7b90222 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:05.742588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939dea0-e607-40d0-bc95-63710fb9ad47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:05.743244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939dea0-e607-40d0-bc95-63710fb9ad47 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.743533 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939dea0-e607-40d0-bc95-63710fb9ad47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.743807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939dea0-e607-40d0-bc95-63710fb9ad47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.744464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939dea0-e607-40d0-bc95-63710fb9ad47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:05.746401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939dea0-e607-40d0-bc95-63710fb9ad47 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:05.787157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1277/2549] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:25:05 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:05.789542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4fdf6-dd05-4a3e-9499-a124a430420f None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:05.808643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4fdf6-dd05-4a3e-9499-a124a430420f None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:05.847313 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:05.882545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40306e50-15e4-4518-9879-83208b9c8c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:05.883217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40306e50-15e4-4518-9879-83208b9c8c97 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.883606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40306e50-15e4-4518-9879-83208b9c8c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.883853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40306e50-15e4-4518-9879-83208b9c8c97 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.884392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40306e50-15e4-4518-9879-83208b9c8c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:05.886583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40306e50-15e4-4518-9879-83208b9c8c97 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:05.911158 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1273/2550] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:25:05 2022] POST /identity/v3/users => generated 295 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:05.915427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939dea0-e607-40d0-bc95-63710fb9ad47 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:05.926069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939dea0-e607-40d0-bc95-63710fb9ad47 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:05.931184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9c857-7b51-4aee-a922-0102ccb2a554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:05.931870 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9c857-7b51-4aee-a922-0102ccb2a554 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.932264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9c857-7b51-4aee-a922-0102ccb2a554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.932545 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9c857-7b51-4aee-a922-0102ccb2a554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.933178 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9c857-7b51-4aee-a922-0102ccb2a554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:05.934776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9c857-7b51-4aee-a922-0102ccb2a554 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:05.946704 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1278/2551] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:05.951168 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40306e50-15e4-4518-9879-83208b9c8c97 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:05.961732 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40306e50-15e4-4518-9879-83208b9c8c97 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:05.967587 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daff3e0-547a-4583-8cea-98b4d3c02aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:05.968265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daff3e0-547a-4583-8cea-98b4d3c02aba admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.968539 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daff3e0-547a-4583-8cea-98b4d3c02aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.968807 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daff3e0-547a-4583-8cea-98b4d3c02aba admin admin] PATH_INFO: `/v3/projects/dbe97c08ec9f485c9f8640bb55fabdb7/users/cbc325444f064ec0907cc0bb0fe6529d/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.986508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daff3e0-547a-4583-8cea-98b4d3c02aba admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbe97c08ec9f485c9f8640bb55fabdb7, user_id=cbc325444f064ec0907cc0bb0fe6529d, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:05.989151 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daff3e0-547a-4583-8cea-98b4d3c02aba admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:06.004043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1274/2552] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:25:05 2022] PUT /identity/v3/projects/dbe97c08ec9f485c9f8640bb55fabdb7/users/cbc325444f064ec0907cc0bb0fe6529d/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:06.008559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9c857-7b51-4aee-a922-0102ccb2a554 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:06.018742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9c857-7b51-4aee-a922-0102ccb2a554 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:06.076034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:06.087736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9020a-4b16-4b83-9dfa-b9ccc696d6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:06.091150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9020a-4b16-4b83-9dfa-b9ccc696d6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:06.091150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9020a-4b16-4b83-9dfa-b9ccc696d6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:06.091150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9020a-4b16-4b83-9dfa-b9ccc696d6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:06.091150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9020a-4b16-4b83-9dfa-b9ccc696d6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:06.091526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9020a-4b16-4b83-9dfa-b9ccc696d6b1 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:06.102389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1279/2553] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:25:06.107046 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daff3e0-547a-4583-8cea-98b4d3c02aba admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:06.120753 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daff3e0-547a-4583-8cea-98b4d3c02aba admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:06.128091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56229f3c-5b73-4a1d-bb72-eb2e8e53975c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:06.128760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56229f3c-5b73-4a1d-bb72-eb2e8e53975c admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:06.129051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56229f3c-5b73-4a1d-bb72-eb2e8e53975c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:06.129309 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56229f3c-5b73-4a1d-bb72-eb2e8e53975c admin admin] PATH_INFO: `/v3/projects/dbe97c08ec9f485c9f8640bb55fabdb7/users/cbc325444f064ec0907cc0bb0fe6529d/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:06.131332 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56229f3c-5b73-4a1d-bb72-eb2e8e53975c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbe97c08ec9f485c9f8640bb55fabdb7, user_id=cbc325444f064ec0907cc0bb0fe6529d, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:06.135172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56229f3c-5b73-4a1d-bb72-eb2e8e53975c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:06.146416 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1275/2554] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:25:06 2022] PUT /identity/v3/projects/dbe97c08ec9f485c9f8640bb55fabdb7/users/cbc325444f064ec0907cc0bb0fe6529d/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:06.157244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4131105-751a-4d6c-8eae-6ab6600ffccc None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:06.157909 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4131105-751a-4d6c-8eae-6ab6600ffccc None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:06.158141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4131105-751a-4d6c-8eae-6ab6600ffccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:06.189201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b4131105-751a-4d6c-8eae-6ab6600ffccc None None] MFA Rules not processed for user `cbc325444f064ec0907cc0bb0fe6529d`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:06.190474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b4131105-751a-4d6c-8eae-6ab6600ffccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:06.291035 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1280/2555] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:06.307554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56229f3c-5b73-4a1d-bb72-eb2e8e53975c admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:06.309509 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-56229f3c-5b73-4a1d-bb72-eb2e8e53975c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:06.374350 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56229f3c-5b73-4a1d-bb72-eb2e8e53975c admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:06.378129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efab3e-92e3-4a8f-ab38-76bd16fdf6e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:06.378814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efab3e-92e3-4a8f-ab38-76bd16fdf6e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:06.379188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efab3e-92e3-4a8f-ab38-76bd16fdf6e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:06.379522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efab3e-92e3-4a8f-ab38-76bd16fdf6e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:06.441506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efab3e-92e3-4a8f-ab38-76bd16fdf6e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:06.442966 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efab3e-92e3-4a8f-ab38-76bd16fdf6e5 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:06.453842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1276/2556] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:12.202219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4131105-751a-4d6c-8eae-6ab6600ffccc None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:12.217938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4131105-751a-4d6c-8eae-6ab6600ffccc None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:12.221936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d278f2-365d-44de-8dc4-4c6e744fedc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:12.222930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d278f2-365d-44de-8dc4-4c6e744fedc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:12.223257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d278f2-365d-44de-8dc4-4c6e744fedc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:12.223540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d278f2-365d-44de-8dc4-4c6e744fedc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:12.234975 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d278f2-365d-44de-8dc4-4c6e744fedc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:12.236633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d278f2-365d-44de-8dc4-4c6e744fedc5 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:12.247962 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1281/2557] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:16.449536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:17.362849 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fa973-6d91-4911-8904-7065618b9a42 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:17.363151 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fa973-6d91-4911-8904-7065618b9a42 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:17.363455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fa973-6d91-4911-8904-7065618b9a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:17.383207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-c54fa973-6d91-4911-8904-7065618b9a42 None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:17.384409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c54fa973-6d91-4911-8904-7065618b9a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:17.411694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d278f2-365d-44de-8dc4-4c6e744fedc5 service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:17.413096 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-56d278f2-365d-44de-8dc4-4c6e744fedc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:17.445461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1277/2558] 10.222.0.61 () {58 vars in 967 bytes} [Wed Sep 7 13:25:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:17.458513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fa973-6d91-4911-8904-7065618b9a42 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:17.460568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c54fa973-6d91-4911-8904-7065618b9a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:17.508744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d278f2-365d-44de-8dc4-4c6e744fedc5 service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:17.514612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf578984-229f-4220-8f79-eb9e5611f4dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:17.515257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf578984-229f-4220-8f79-eb9e5611f4dd service placement] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:17.515544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf578984-229f-4220-8f79-eb9e5611f4dd service placement] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:17.515821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf578984-229f-4220-8f79-eb9e5611f4dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:17.517585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bf578984-229f-4220-8f79-eb9e5611f4dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:17.530458 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf578984-229f-4220-8f79-eb9e5611f4dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:17.532030 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fa973-6d91-4911-8904-7065618b9a42 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:17.532268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf578984-229f-4220-8f79-eb9e5611f4dd service placement] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:17.537347 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1c413-2679-40de-b5f5-202b79f494ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:17.537975 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1c413-2679-40de-b5f5-202b79f494ba service swift] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:17.538311 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1c413-2679-40de-b5f5-202b79f494ba service swift] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:17.539143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1c413-2679-40de-b5f5-202b79f494ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:17.543614 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1282/2559] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:25:17.551854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1c413-2679-40de-b5f5-202b79f494ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:17.553537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1c413-2679-40de-b5f5-202b79f494ba service swift] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:17.562472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1278/2560] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:25:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 13:25:27.540242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=84 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:27.559151 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=57 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:38.430505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf578984-229f-4220-8f79-eb9e5611f4dd service placement] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:38.499626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf578984-229f-4220-8f79-eb9e5611f4dd service placement] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:38.508378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:38.509021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:38.509296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:38.509567 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] PATH_INFO: `/v3/users/2ab6c4f021594060bb18d0ba9692d1a6` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:38.510699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab6c4f021594060bb18d0ba9692d1a6)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:38.512671 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:38.559876 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab6c4f021594060bb18d0ba9692d1a6'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:38.571847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab6c4f021594060bb18d0ba9692d1a6'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:38.578589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab6c4f021594060bb18d0ba9692d1a6'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:38.579382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab6c4f021594060bb18d0ba9692d1a6'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:38.579987 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab6c4f021594060bb18d0ba9692d1a6'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:38.592882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1283/2561] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:25:38 2022] DELETE /identity/v3/users/2ab6c4f021594060bb18d0ba9692d1a6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:38.597394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00779603-0ed9-48f8-8d21-e2e6d832b55e None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:38.598151 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00779603-0ed9-48f8-8d21-e2e6d832b55e None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:38.598151 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00779603-0ed9-48f8-8d21-e2e6d832b55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:38.617525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-00779603-0ed9-48f8-8d21-e2e6d832b55e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:38.618712 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-00779603-0ed9-48f8-8d21-e2e6d832b55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:38.678867 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1279/2562] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:25:38.689180 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:38.690626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:38.748277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec627-6ef5-474b-a3fa-79ad7b479155 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:38.751987 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acf451-e168-4381-80ec-41f4e45d20db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:38.752595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acf451-e168-4381-80ec-41f4e45d20db service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:38.752878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acf451-e168-4381-80ec-41f4e45d20db service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:38.753156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acf451-e168-4381-80ec-41f4e45d20db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:38.762276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acf451-e168-4381-80ec-41f4e45d20db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:38.763749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acf451-e168-4381-80ec-41f4e45d20db service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:38.773653 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1284/2563] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:39.290986 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00779603-0ed9-48f8-8d21-e2e6d832b55e None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:39.293003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-00779603-0ed9-48f8-8d21-e2e6d832b55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:39.306757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00779603-0ed9-48f8-8d21-e2e6d832b55e None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:39.314661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:39.315510 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:39.316054 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:39.316433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] PATH_INFO: `/v3/projects/cf35a719d9774906a8a073362cfb4d88` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:39.318387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf35a719d9774906a8a073362cfb4d88)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:39.321015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:39.396323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf35a719d9774906a8a073362cfb4d88', 'request_id': 'req-4be6acbb-e837-4819-9af0-32527c6385c3'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:39.404281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf35a719d9774906a8a073362cfb4d88', 'request_id': 'req-4be6acbb-e837-4819-9af0-32527c6385c3'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:39.410545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf35a719d9774906a8a073362cfb4d88', 'request_id': 'req-4be6acbb-e837-4819-9af0-32527c6385c3'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:39.422569 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] The token cache is being invalidate because project cf35a719d9774906a8a073362cfb4d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:25:39.422854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:39.423524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:39.425071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1280/2564] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:25:39 2022] DELETE /identity/v3/projects/cf35a719d9774906a8a073362cfb4d88 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:25:39.429056 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acf451-e168-4381-80ec-41f4e45d20db service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:39.439822 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acf451-e168-4381-80ec-41f4e45d20db service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:39.460987 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:39.513401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f17546-b796-435a-95c1-b0a12c49e21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:39.514072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f17546-b796-435a-95c1-b0a12c49e21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:39.514353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f17546-b796-435a-95c1-b0a12c49e21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:39.514771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f17546-b796-435a-95c1-b0a12c49e21c admin admin] PATH_INFO: `/v3/users/cbc325444f064ec0907cc0bb0fe6529d` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:39.516096 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f17546-b796-435a-95c1-b0a12c49e21c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbc325444f064ec0907cc0bb0fe6529d)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:39.518045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f17546-b796-435a-95c1-b0a12c49e21c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:39.574618 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f9f17546-b796-435a-95c1-b0a12c49e21c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbc325444f064ec0907cc0bb0fe6529d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:39.597634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f9f17546-b796-435a-95c1-b0a12c49e21c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbc325444f064ec0907cc0bb0fe6529d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:39.605638 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f9f17546-b796-435a-95c1-b0a12c49e21c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbc325444f064ec0907cc0bb0fe6529d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:39.606483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f9f17546-b796-435a-95c1-b0a12c49e21c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbc325444f064ec0907cc0bb0fe6529d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:39.607072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-f9f17546-b796-435a-95c1-b0a12c49e21c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbc325444f064ec0907cc0bb0fe6529d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:39.621129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1285/2565] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:25:39 2022] DELETE /identity/v3/users/cbc325444f064ec0907cc0bb0fe6529d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:39.986777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:39.990040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:40.009865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6acbb-e837-4819-9af0-32527c6385c3 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:40.119265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:40.119954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:40.120240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:40.120515 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97 admin admin] PATH_INFO: `/v3/projects/dbe97c08ec9f485c9f8640bb55fabdb7` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:40.121669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbe97c08ec9f485c9f8640bb55fabdb7)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:40.123887 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:40.162626 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:40.186789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbe97c08ec9f485c9f8640bb55fabdb7', 'request_id': 'req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:40.196828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbe97c08ec9f485c9f8640bb55fabdb7', 'request_id': 'req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:40.203921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbe97c08ec9f485c9f8640bb55fabdb7', 'request_id': 'req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:40.220055 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97 admin admin] The token cache is being invalidate because project dbe97c08ec9f485c9f8640bb55fabdb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:25:40.220356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:40.221027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-dca4eed3-c37e-4ea5-8903-af5a6bf1af97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:40.222523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1281/2566] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:25:39 2022] DELETE /identity/v3/projects/dbe97c08ec9f485c9f8640bb55fabdb7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:40.227516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f65037-18a0-4a30-a04a-3151073c41e4 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:40.227839 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f65037-18a0-4a30-a04a-3151073c41e4 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:40.228115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f65037-18a0-4a30-a04a-3151073c41e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:40.252312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-75f65037-18a0-4a30-a04a-3151073c41e4 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:40.253464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-75f65037-18a0-4a30-a04a-3151073c41e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:40.322946 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1286/2567] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:40.333283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1442ac-c47b-4e0f-a895-c1c5aa37f742 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:40.333735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1442ac-c47b-4e0f-a895-c1c5aa37f742 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:40.334076 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1442ac-c47b-4e0f-a895-c1c5aa37f742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:40.364898 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ad1442ac-c47b-4e0f-a895-c1c5aa37f742 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:40.366193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ad1442ac-c47b-4e0f-a895-c1c5aa37f742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:40.376951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1282/2568] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:40.382889 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f65037-18a0-4a30-a04a-3151073c41e4 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:40.398077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f65037-18a0-4a30-a04a-3151073c41e4 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:40.403264 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fc115-eda9-49ff-a13e-02355b03ce3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:40.404119 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fc115-eda9-49ff-a13e-02355b03ce3c admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:40.404406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fc115-eda9-49ff-a13e-02355b03ce3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:40.404676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fc115-eda9-49ff-a13e-02355b03ce3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:40.405337 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fc115-eda9-49ff-a13e-02355b03ce3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:25:40.405644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fc115-eda9-49ff-a13e-02355b03ce3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:40.407610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fc115-eda9-49ff-a13e-02355b03ce3c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:40.421057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1287/2569] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:25:40.428346 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1442ac-c47b-4e0f-a895-c1c5aa37f742 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:40.449441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1442ac-c47b-4e0f-a895-c1c5aa37f742 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:40.455155 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03babff1-971c-48da-9a3f-0ae1b777853d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:40.455519 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03babff1-971c-48da-9a3f-0ae1b777853d admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:40.455839 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03babff1-971c-48da-9a3f-0ae1b777853d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:40.456099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03babff1-971c-48da-9a3f-0ae1b777853d admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:40.456763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03babff1-971c-48da-9a3f-0ae1b777853d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:40.458673 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03babff1-971c-48da-9a3f-0ae1b777853d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:40.490243 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1283/2570] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:25:40 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:40.494506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fc115-eda9-49ff-a13e-02355b03ce3c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:40.506020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fc115-eda9-49ff-a13e-02355b03ce3c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:40.540727 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:40.583637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbc9f6-2ae0-44ca-9832-63df8bfaedb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:40.584574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbc9f6-2ae0-44ca-9832-63df8bfaedb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:40.585761 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbc9f6-2ae0-44ca-9832-63df8bfaedb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:40.586242 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbc9f6-2ae0-44ca-9832-63df8bfaedb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:40.587069 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbc9f6-2ae0-44ca-9832-63df8bfaedb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:40.591930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbc9f6-2ae0-44ca-9832-63df8bfaedb4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:40.629616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1288/2571] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:25:40 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:40.633780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03babff1-971c-48da-9a3f-0ae1b777853d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:40.644954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03babff1-971c-48da-9a3f-0ae1b777853d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:40.650074 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb82fca-d39f-47ed-8ac0-b276472fef01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:40.650698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82fca-d39f-47ed-8ac0-b276472fef01 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:40.650982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82fca-d39f-47ed-8ac0-b276472fef01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:40.651433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82fca-d39f-47ed-8ac0-b276472fef01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:40.652096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb82fca-d39f-47ed-8ac0-b276472fef01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:40.653733 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb82fca-d39f-47ed-8ac0-b276472fef01 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:40.667888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1284/2572] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:40.671584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbc9f6-2ae0-44ca-9832-63df8bfaedb4 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:40.683984 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbc9f6-2ae0-44ca-9832-63df8bfaedb4 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:40.689349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80567f1a-ec5b-4f36-8ec4-7163f22dd11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:40.690204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80567f1a-ec5b-4f36-8ec4-7163f22dd11d admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:40.690350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80567f1a-ec5b-4f36-8ec4-7163f22dd11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:40.690540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80567f1a-ec5b-4f36-8ec4-7163f22dd11d admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:40.691181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80567f1a-ec5b-4f36-8ec4-7163f22dd11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:40.692796 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80567f1a-ec5b-4f36-8ec4-7163f22dd11d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:40.707191 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1289/2573] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:40.711203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb82fca-d39f-47ed-8ac0-b276472fef01 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:40.723472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb82fca-d39f-47ed-8ac0-b276472fef01 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:40.728438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1095d3-11ee-45f5-bcf8-4674c380b999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:40.729189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1095d3-11ee-45f5-bcf8-4674c380b999 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:40.730250 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1095d3-11ee-45f5-bcf8-4674c380b999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:40.730528 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1095d3-11ee-45f5-bcf8-4674c380b999 admin admin] PATH_INFO: `/v3/projects/f35ece6b446241009350b34d8becaf1e/users/58d65b0fca3f4a5390097f6b776f6167/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:40.747765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1095d3-11ee-45f5-bcf8-4674c380b999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f35ece6b446241009350b34d8becaf1e, user_id=58d65b0fca3f4a5390097f6b776f6167, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:40.750757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1095d3-11ee-45f5-bcf8-4674c380b999 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:40.769014 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1285/2574] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:25:40 2022] PUT /identity/v3/projects/f35ece6b446241009350b34d8becaf1e/users/58d65b0fca3f4a5390097f6b776f6167/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:40.780327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efba7a-b85f-493f-95d8-91a7a599654d None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:40.780757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efba7a-b85f-493f-95d8-91a7a599654d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:40.781007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efba7a-b85f-493f-95d8-91a7a599654d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:40.820934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f5efba7a-b85f-493f-95d8-91a7a599654d None None] MFA Rules not processed for user `58d65b0fca3f4a5390097f6b776f6167`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:40.822259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f5efba7a-b85f-493f-95d8-91a7a599654d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:40.900630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1290/2575] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:40.905385 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1095d3-11ee-45f5-bcf8-4674c380b999 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:40.974826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1095d3-11ee-45f5-bcf8-4674c380b999 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:40.980551 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc53111-4fee-4749-8390-4adcbe50d7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:40.981035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc53111-4fee-4749-8390-4adcbe50d7f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:40.981313 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc53111-4fee-4749-8390-4adcbe50d7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:40.981585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc53111-4fee-4749-8390-4adcbe50d7f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:40.982299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc53111-4fee-4749-8390-4adcbe50d7f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:40.984357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc53111-4fee-4749-8390-4adcbe50d7f4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:41.016613 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1286/2576] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:25:40 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:41.020833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efba7a-b85f-493f-95d8-91a7a599654d None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:41.033520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efba7a-b85f-493f-95d8-91a7a599654d None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:41.115418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cf50c-433a-4932-8d44-a22604aaa7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:41.116059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cf50c-433a-4932-8d44-a22604aaa7dd admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:41.116344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cf50c-433a-4932-8d44-a22604aaa7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:41.116683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cf50c-433a-4932-8d44-a22604aaa7dd admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:41.117338 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cf50c-433a-4932-8d44-a22604aaa7dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:41.119509 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cf50c-433a-4932-8d44-a22604aaa7dd admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:41.147366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1291/2577] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:25:41 2022] POST /identity/v3/users => generated 296 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:41.148669 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc53111-4fee-4749-8390-4adcbe50d7f4 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:41.159630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc53111-4fee-4749-8390-4adcbe50d7f4 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:41.164312 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a26d95-cdec-43f9-b4e4-61b913375c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:41.164927 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a26d95-cdec-43f9-b4e4-61b913375c7e admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:41.165213 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a26d95-cdec-43f9-b4e4-61b913375c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:41.165485 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a26d95-cdec-43f9-b4e4-61b913375c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:41.166131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a26d95-cdec-43f9-b4e4-61b913375c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:41.167789 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a26d95-cdec-43f9-b4e4-61b913375c7e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:41.180686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1287/2578] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:41.185551 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cf50c-433a-4932-8d44-a22604aaa7dd admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:41.196494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cf50c-433a-4932-8d44-a22604aaa7dd admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:41.202283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1cb5f-080f-4c6b-99da-adf12b6cb8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:41.202951 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1cb5f-080f-4c6b-99da-adf12b6cb8db admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:41.203288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1cb5f-080f-4c6b-99da-adf12b6cb8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:41.203568 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1cb5f-080f-4c6b-99da-adf12b6cb8db admin admin] PATH_INFO: `/v3/projects/048261943d884645bab46ca8a4322415/users/7e3b287374804fb9a8ccef8f3856ec43/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:41.221903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1cb5f-080f-4c6b-99da-adf12b6cb8db admin admin] RBAC: Authorizing `identity:create_grant(project_id=048261943d884645bab46ca8a4322415, user_id=7e3b287374804fb9a8ccef8f3856ec43, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:41.224724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1cb5f-080f-4c6b-99da-adf12b6cb8db admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:41.239486 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1292/2579] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:25:41 2022] PUT /identity/v3/projects/048261943d884645bab46ca8a4322415/users/7e3b287374804fb9a8ccef8f3856ec43/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:41.243655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a26d95-cdec-43f9-b4e4-61b913375c7e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:41.254348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a26d95-cdec-43f9-b4e4-61b913375c7e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:41.274645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:41.315614 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431698c-a77b-45f7-b32e-52ced83c6a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:41.316220 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431698c-a77b-45f7-b32e-52ced83c6a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:41.316501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431698c-a77b-45f7-b32e-52ced83c6a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:41.316769 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431698c-a77b-45f7-b32e-52ced83c6a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:41.317413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431698c-a77b-45f7-b32e-52ced83c6a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:41.319016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431698c-a77b-45f7-b32e-52ced83c6a0b admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:41.329697 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1288/2580] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:41.334107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1cb5f-080f-4c6b-99da-adf12b6cb8db admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:41.341525 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:41.348249 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1cb5f-080f-4c6b-99da-adf12b6cb8db admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:41.353944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1aeb4-9abd-4037-80b2-9725297ec55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:41.355390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1aeb4-9abd-4037-80b2-9725297ec55f admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:41.355668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1aeb4-9abd-4037-80b2-9725297ec55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:41.355940 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1aeb4-9abd-4037-80b2-9725297ec55f admin admin] PATH_INFO: `/v3/projects/048261943d884645bab46ca8a4322415/users/7e3b287374804fb9a8ccef8f3856ec43/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:41.358053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1aeb4-9abd-4037-80b2-9725297ec55f admin admin] RBAC: Authorizing `identity:create_grant(project_id=048261943d884645bab46ca8a4322415, user_id=7e3b287374804fb9a8ccef8f3856ec43, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:41.360719 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1aeb4-9abd-4037-80b2-9725297ec55f admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:41.372428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1293/2581] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:25:41 2022] PUT /identity/v3/projects/048261943d884645bab46ca8a4322415/users/7e3b287374804fb9a8ccef8f3856ec43/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:41.384221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941978c-eaff-48f0-bfda-ed5b16e38861 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:41.384577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941978c-eaff-48f0-bfda-ed5b16e38861 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:41.385034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941978c-eaff-48f0-bfda-ed5b16e38861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:41.413180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-7941978c-eaff-48f0-bfda-ed5b16e38861 None None] MFA Rules not processed for user `7e3b287374804fb9a8ccef8f3856ec43`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:41.414412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7941978c-eaff-48f0-bfda-ed5b16e38861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:41.487462 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1289/2582] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:41.505966 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1aeb4-9abd-4037-80b2-9725297ec55f admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:41.508042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-cde1aeb4-9abd-4037-80b2-9725297ec55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:41.572173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1aeb4-9abd-4037-80b2-9725297ec55f admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:41.576489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930bca1-860a-4580-8d13-a811be7b6e70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:41.577218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930bca1-860a-4580-8d13-a811be7b6e70 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:41.577335 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930bca1-860a-4580-8d13-a811be7b6e70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:41.577578 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930bca1-860a-4580-8d13-a811be7b6e70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:41.589670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930bca1-860a-4580-8d13-a811be7b6e70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:41.591689 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930bca1-860a-4580-8d13-a811be7b6e70 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:41.601736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1294/2583] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:25:41.997636 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941978c-eaff-48f0-bfda-ed5b16e38861 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:42.007841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941978c-eaff-48f0-bfda-ed5b16e38861 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:42.012594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5dc42-775f-4268-b077-096ecb6ae836 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e3b287374804fb9a8ccef8f3856ec43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '048261943d884645bab46ca8a4322415', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:42.013205 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5dc42-775f-4268-b077-096ecb6ae836 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:42.013484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5dc42-775f-4268-b077-096ecb6ae836 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:42.013763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5dc42-775f-4268-b077-096ecb6ae836 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:42.014414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5dc42-775f-4268-b077-096ecb6ae836 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:25:42.014717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5dc42-775f-4268-b077-096ecb6ae836 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:42.016352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5dc42-775f-4268-b077-096ecb6ae836 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:42.035307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1290/2584] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:25:42.043280 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930bca1-860a-4580-8d13-a811be7b6e70 service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:42.054363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930bca1-860a-4580-8d13-a811be7b6e70 service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:42.058982 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc0968-79b0-4ab6-b7dc-482d7632425b tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e3b287374804fb9a8ccef8f3856ec43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '048261943d884645bab46ca8a4322415', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:42.059620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc0968-79b0-4ab6-b7dc-482d7632425b tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:42.059900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc0968-79b0-4ab6-b7dc-482d7632425b tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:42.060171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc0968-79b0-4ab6-b7dc-482d7632425b tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:42.060797 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc0968-79b0-4ab6-b7dc-482d7632425b tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:42.062742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc0968-79b0-4ab6-b7dc-482d7632425b tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:42.101172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1295/2585] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:25:42 2022] POST /identity/v3/projects => generated 355 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:42.105983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5dc42-775f-4268-b077-096ecb6ae836 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:42.117075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5dc42-775f-4268-b077-096ecb6ae836 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:42.180856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381b0b9-1560-441d-8522-1684a70e84c3 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e3b287374804fb9a8ccef8f3856ec43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '048261943d884645bab46ca8a4322415', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:42.181479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381b0b9-1560-441d-8522-1684a70e84c3 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:42.181766 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381b0b9-1560-441d-8522-1684a70e84c3 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:42.182045 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381b0b9-1560-441d-8522-1684a70e84c3 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:42.182751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381b0b9-1560-441d-8522-1684a70e84c3 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:25:42.183063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381b0b9-1560-441d-8522-1684a70e84c3 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:42.184882 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381b0b9-1560-441d-8522-1684a70e84c3 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:42.199077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1291/2586] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:25:42.240910 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc0968-79b0-4ab6-b7dc-482d7632425b tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:42.252791 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc0968-79b0-4ab6-b7dc-482d7632425b tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:42.258076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2202a64-33e2-4ea1-b24b-535ac3e2c872 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e3b287374804fb9a8ccef8f3856ec43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '048261943d884645bab46ca8a4322415', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:42.258714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2202a64-33e2-4ea1-b24b-535ac3e2c872 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:42.259048 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2202a64-33e2-4ea1-b24b-535ac3e2c872 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:42.259409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2202a64-33e2-4ea1-b24b-535ac3e2c872 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] PATH_INFO: `/v3/projects/f0657c71f95f4a94b1a9df9c7c18de0c` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:42.261167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2202a64-33e2-4ea1-b24b-535ac3e2c872 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f0657c71f95f4a94b1a9df9c7c18de0c)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:42.263222 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2202a64-33e2-4ea1-b24b-535ac3e2c872 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:42.313738 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a2202a64-33e2-4ea1-b24b-535ac3e2c872 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0657c71f95f4a94b1a9df9c7c18de0c', 'request_id': 'req-a2202a64-33e2-4ea1-b24b-535ac3e2c872'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:42.320938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a2202a64-33e2-4ea1-b24b-535ac3e2c872 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0657c71f95f4a94b1a9df9c7c18de0c', 'request_id': 'req-a2202a64-33e2-4ea1-b24b-535ac3e2c872'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:42.330256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a2202a64-33e2-4ea1-b24b-535ac3e2c872 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0657c71f95f4a94b1a9df9c7c18de0c', 'request_id': 'req-a2202a64-33e2-4ea1-b24b-535ac3e2c872'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:42.346496 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a2202a64-33e2-4ea1-b24b-535ac3e2c872 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] The token cache is being invalidate because project f0657c71f95f4a94b1a9df9c7c18de0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:25:42.346788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a2202a64-33e2-4ea1-b24b-535ac3e2c872 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:42.347480 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-a2202a64-33e2-4ea1-b24b-535ac3e2c872 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:42.349347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1296/2587] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:25:42 2022] DELETE /identity/v3/projects/f0657c71f95f4a94b1a9df9c7c18de0c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:42.542631 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381b0b9-1560-441d-8522-1684a70e84c3 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:42.544467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d381b0b9-1560-441d-8522-1684a70e84c3 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:42.576068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:42.610870 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381b0b9-1560-441d-8522-1684a70e84c3 tempest-VolumeQuotaClassesTest-278316189 tempest-VolumeQuotaClassesTest-278316189-project-admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:42.673984 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:42.674635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:42.674925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:42.675227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] PATH_INFO: `/v3/users/58d65b0fca3f4a5390097f6b776f6167` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:42.676447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58d65b0fca3f4a5390097f6b776f6167)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:42.678497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:42.737127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58d65b0fca3f4a5390097f6b776f6167'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:42.744702 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58d65b0fca3f4a5390097f6b776f6167'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:42.745369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58d65b0fca3f4a5390097f6b776f6167'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:42.745982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58d65b0fca3f4a5390097f6b776f6167'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:42.759068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58d65b0fca3f4a5390097f6b776f6167'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:42.772851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1292/2588] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:25:42 2022] DELETE /identity/v3/users/58d65b0fca3f4a5390097f6b776f6167 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:42.778578 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c31a0-2664-4153-a6c7-a6d0b972b6dd None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:42.779152 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c31a0-2664-4153-a6c7-a6d0b972b6dd None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:42.779277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c31a0-2664-4153-a6c7-a6d0b972b6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:42.801554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-745c31a0-2664-4153-a6c7-a6d0b972b6dd None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:42.802946 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-745c31a0-2664-4153-a6c7-a6d0b972b6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:42.861973 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:42.875677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1297/2589] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:42.889153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:42.890609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:42.951183 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ccceb-e96c-490c-8eb9-39cd4b18c2e1 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:42.956033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6fe81-4910-4c2a-baf1-3d733387e053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:42.956649 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6fe81-4910-4c2a-baf1-3d733387e053 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:42.956927 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6fe81-4910-4c2a-baf1-3d733387e053 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:42.957196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6fe81-4910-4c2a-baf1-3d733387e053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:42.964496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:42.967652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6fe81-4910-4c2a-baf1-3d733387e053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:42.970077 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6fe81-4910-4c2a-baf1-3d733387e053 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:42.981132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1293/2590] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:25:43.264862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c31a0-2664-4153-a6c7-a6d0b972b6dd None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:43.266357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-745c31a0-2664-4153-a6c7-a6d0b972b6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:43.278888 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c31a0-2664-4153-a6c7-a6d0b972b6dd None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:43.283756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:43.284418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:43.284686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:43.284960 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] PATH_INFO: `/v3/projects/f35ece6b446241009350b34d8becaf1e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:43.286315 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f35ece6b446241009350b34d8becaf1e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:43.288296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:43.340551 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f35ece6b446241009350b34d8becaf1e', 'request_id': 'req-dc240184-10f8-4d23-bd31-15c64d842a43'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.348541 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f35ece6b446241009350b34d8becaf1e', 'request_id': 'req-dc240184-10f8-4d23-bd31-15c64d842a43'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.355248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f35ece6b446241009350b34d8becaf1e', 'request_id': 'req-dc240184-10f8-4d23-bd31-15c64d842a43'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.368990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] The token cache is being invalidate because project f35ece6b446241009350b34d8becaf1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:25:43.369352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.370419 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.372498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1298/2591] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:25:43 2022] DELETE /identity/v3/projects/f35ece6b446241009350b34d8becaf1e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:43.376604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6fe81-4910-4c2a-baf1-3d733387e053 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:43.387574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6fe81-4910-4c2a-baf1-3d733387e053 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:43.452528 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004534a-1387-40da-a38c-bec5398b1dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:43.453209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004534a-1387-40da-a38c-bec5398b1dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:43.453610 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004534a-1387-40da-a38c-bec5398b1dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:43.453791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004534a-1387-40da-a38c-bec5398b1dd8 admin admin] PATH_INFO: `/v3/users/7e3b287374804fb9a8ccef8f3856ec43` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:43.454936 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004534a-1387-40da-a38c-bec5398b1dd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e3b287374804fb9a8ccef8f3856ec43)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:43.459373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004534a-1387-40da-a38c-bec5398b1dd8 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:43.508195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e004534a-1387-40da-a38c-bec5398b1dd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e3b287374804fb9a8ccef8f3856ec43'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.516316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e004534a-1387-40da-a38c-bec5398b1dd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e3b287374804fb9a8ccef8f3856ec43'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.516928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e004534a-1387-40da-a38c-bec5398b1dd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e3b287374804fb9a8ccef8f3856ec43'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.517532 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e004534a-1387-40da-a38c-bec5398b1dd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e3b287374804fb9a8ccef8f3856ec43'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.529652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-e004534a-1387-40da-a38c-bec5398b1dd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e3b287374804fb9a8ccef8f3856ec43'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.541908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1294/2592] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:25:43 2022] DELETE /identity/v3/users/7e3b287374804fb9a8ccef8f3856ec43 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:43.834225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:43.835939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:43.851014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc240184-10f8-4d23-bd31-15c64d842a43 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:43.913593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:43.914157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:43.914445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:43.914717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b admin admin] PATH_INFO: `/v3/projects/048261943d884645bab46ca8a4322415` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:43.915900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=048261943d884645bab46ca8a4322415)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:43.918596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:43.971285 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048261943d884645bab46ca8a4322415', 'request_id': 'req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.977726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048261943d884645bab46ca8a4322415', 'request_id': 'req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.984315 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048261943d884645bab46ca8a4322415', 'request_id': 'req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.997057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b admin admin] The token cache is being invalidate because project 048261943d884645bab46ca8a4322415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:25:43.997342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.998157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8f8d132a-8ebd-478d-9ff1-38bce646aa3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:43.999751 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1299/2593] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:25:43 2022] DELETE /identity/v3/projects/048261943d884645bab46ca8a4322415 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:44.004786 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e047da-eca8-49d6-85c1-9d8471cd94da None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:44.004982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e047da-eca8-49d6-85c1-9d8471cd94da None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:44.005257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e047da-eca8-49d6-85c1-9d8471cd94da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:44.025668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-e6e047da-eca8-49d6-85c1-9d8471cd94da None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:44.028601 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e6e047da-eca8-49d6-85c1-9d8471cd94da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:44.097262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1295/2594] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:25:44.105522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca779b-583b-4211-a979-12e0728b733b None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:44.105894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca779b-583b-4211-a979-12e0728b733b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:44.106234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca779b-583b-4211-a979-12e0728b733b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:44.121780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:44.129187 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-30ca779b-583b-4211-a979-12e0728b733b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:44.131191 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-30ca779b-583b-4211-a979-12e0728b733b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:44.140027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1300/2595] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:44.144375 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e047da-eca8-49d6-85c1-9d8471cd94da None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:44.154316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e047da-eca8-49d6-85c1-9d8471cd94da None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:44.158999 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447b8b2-282f-4c49-b02b-8ff3b65d3e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:44.159639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447b8b2-282f-4c49-b02b-8ff3b65d3e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:44.160003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447b8b2-282f-4c49-b02b-8ff3b65d3e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:44.160407 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447b8b2-282f-4c49-b02b-8ff3b65d3e54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:44.161117 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447b8b2-282f-4c49-b02b-8ff3b65d3e54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:25:44.161504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447b8b2-282f-4c49-b02b-8ff3b65d3e54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:44.163148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447b8b2-282f-4c49-b02b-8ff3b65d3e54 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:44.175413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1296/2596] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:25:44.179758 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca779b-583b-4211-a979-12e0728b733b None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:44.189588 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca779b-583b-4211-a979-12e0728b733b None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:44.194351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ebb1f-cae8-4e47-8805-97521043ce5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:44.194958 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ebb1f-cae8-4e47-8805-97521043ce5a admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:44.195285 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ebb1f-cae8-4e47-8805-97521043ce5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:44.195563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ebb1f-cae8-4e47-8805-97521043ce5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:44.196210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ebb1f-cae8-4e47-8805-97521043ce5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:44.198700 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ebb1f-cae8-4e47-8805-97521043ce5a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:44.226546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1301/2597] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:25:44 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:44.231131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447b8b2-282f-4c49-b02b-8ff3b65d3e54 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:44.237238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:44.243354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447b8b2-282f-4c49-b02b-8ff3b65d3e54 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:44.302069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac73726-ace4-4c04-80c8-81431e861f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:44.302680 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac73726-ace4-4c04-80c8-81431e861f98 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:44.302962 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac73726-ace4-4c04-80c8-81431e861f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:44.303267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac73726-ace4-4c04-80c8-81431e861f98 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:44.303901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac73726-ace4-4c04-80c8-81431e861f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:44.305831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac73726-ace4-4c04-80c8-81431e861f98 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:44.328917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1297/2598] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:25:44 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:25:44.332638 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ebb1f-cae8-4e47-8805-97521043ce5a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:44.345691 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ebb1f-cae8-4e47-8805-97521043ce5a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:44.350674 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd4f09-3533-4b0e-9a48-49571bd54ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:44.351314 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd4f09-3533-4b0e-9a48-49571bd54ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:44.351602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd4f09-3533-4b0e-9a48-49571bd54ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:44.351884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd4f09-3533-4b0e-9a48-49571bd54ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:44.352536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd4f09-3533-4b0e-9a48-49571bd54ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:44.354143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd4f09-3533-4b0e-9a48-49571bd54ca4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:44.365860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1302/2599] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:44.370027 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac73726-ace4-4c04-80c8-81431e861f98 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:44.380813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac73726-ace4-4c04-80c8-81431e861f98 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:44.385638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fbac4-a22c-4606-9630-4327a3183458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:44.386242 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fbac4-a22c-4606-9630-4327a3183458 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:44.386524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fbac4-a22c-4606-9630-4327a3183458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:44.386799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fbac4-a22c-4606-9630-4327a3183458 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:44.387486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fbac4-a22c-4606-9630-4327a3183458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:44.389102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fbac4-a22c-4606-9630-4327a3183458 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:44.401477 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1298/2600] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:44.406697 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd4f09-3533-4b0e-9a48-49571bd54ca4 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:44.417402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd4f09-3533-4b0e-9a48-49571bd54ca4 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:44.421990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68253c4-81f4-4d0e-bc06-bcd59535f20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:44.422715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68253c4-81f4-4d0e-bc06-bcd59535f20d admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:44.422992 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68253c4-81f4-4d0e-bc06-bcd59535f20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:44.423309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68253c4-81f4-4d0e-bc06-bcd59535f20d admin admin] PATH_INFO: `/v3/projects/676fb0fe414e4fff9e943caa9f50343a/users/4da54393548b4921a9e072a75bd24523/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:44.439774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68253c4-81f4-4d0e-bc06-bcd59535f20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=676fb0fe414e4fff9e943caa9f50343a, user_id=4da54393548b4921a9e072a75bd24523, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:44.442389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68253c4-81f4-4d0e-bc06-bcd59535f20d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:44.456266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1303/2601] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:25:44 2022] PUT /identity/v3/projects/676fb0fe414e4fff9e943caa9f50343a/users/4da54393548b4921a9e072a75bd24523/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:44.468387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ef91e-b983-4821-a4ab-2edef8d44b7b None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:44.468722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ef91e-b983-4821-a4ab-2edef8d44b7b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:44.469363 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ef91e-b983-4821-a4ab-2edef8d44b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:44.501599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-de7ef91e-b983-4821-a4ab-2edef8d44b7b None None] MFA Rules not processed for user `4da54393548b4921a9e072a75bd24523`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:44.504032 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-de7ef91e-b983-4821-a4ab-2edef8d44b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:44.593345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1299/2602] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:44.607040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68253c4-81f4-4d0e-bc06-bcd59535f20d admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:44.608021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b68253c4-81f4-4d0e-bc06-bcd59535f20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:44.673473 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68253c4-81f4-4d0e-bc06-bcd59535f20d admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:44.677593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b07e3-20ee-435a-9d6d-bd35cf61b1cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:44.678240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b07e3-20ee-435a-9d6d-bd35cf61b1cf service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:44.678526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b07e3-20ee-435a-9d6d-bd35cf61b1cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:44.678932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b07e3-20ee-435a-9d6d-bd35cf61b1cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:44.693349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b07e3-20ee-435a-9d6d-bd35cf61b1cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:44.694690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b07e3-20ee-435a-9d6d-bd35cf61b1cf service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:44.704075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1304/2603] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:45.801842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ef91e-b983-4821-a4ab-2edef8d44b7b None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:45.804776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-de7ef91e-b983-4821-a4ab-2edef8d44b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:45.869000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ef91e-b983-4821-a4ab-2edef8d44b7b None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:45.872757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe0ede-944d-416c-bd27-3ed74e6d4d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:45.873373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe0ede-944d-416c-bd27-3ed74e6d4d10 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:45.873653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe0ede-944d-416c-bd27-3ed74e6d4d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:45.873926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe0ede-944d-416c-bd27-3ed74e6d4d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:45.875805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-dfbe0ede-944d-416c-bd27-3ed74e6d4d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:45.907057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:45.947750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe0ede-944d-416c-bd27-3ed74e6d4d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:45.949818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe0ede-944d-416c-bd27-3ed74e6d4d10 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:45.961567 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1300/2604] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:25:47.301479 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b07e3-20ee-435a-9d6d-bd35cf61b1cf service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:47.313919 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b07e3-20ee-435a-9d6d-bd35cf61b1cf service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:47.319548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:47.320519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:47.320715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:47.320920 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] PATH_INFO: `/v3/users/4da54393548b4921a9e072a75bd24523` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:47.322924 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=4da54393548b4921a9e072a75bd24523)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:47.325573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:47.391824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4da54393548b4921a9e072a75bd24523'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:47.411855 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4da54393548b4921a9e072a75bd24523'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:47.423893 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4da54393548b4921a9e072a75bd24523'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:47.424934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4da54393548b4921a9e072a75bd24523'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:47.425896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4da54393548b4921a9e072a75bd24523'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:47.443554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1305/2605] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:25:47 2022] DELETE /identity/v3/users/4da54393548b4921a9e072a75bd24523 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:47.451784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ebc61-7350-44b1-b4be-e1b57234c61c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:47.452291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ebc61-7350-44b1-b4be-e1b57234c61c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:47.452389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ebc61-7350-44b1-b4be-e1b57234c61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:47.474796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-185ebc61-7350-44b1-b4be-e1b57234c61c None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:47.476140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-185ebc61-7350-44b1-b4be-e1b57234c61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:47.557655 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1301/2606] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:47.568633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:47.570950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:47.592031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:47.640269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5aad1f-7a7b-4a28-ba46-8a89732013ab admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:47.644522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47590403-8d04-4b2c-bd1b-69589efc7295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:47.645276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47590403-8d04-4b2c-bd1b-69589efc7295 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:47.645618 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47590403-8d04-4b2c-bd1b-69589efc7295 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:47.645972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47590403-8d04-4b2c-bd1b-69589efc7295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:47.658985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47590403-8d04-4b2c-bd1b-69589efc7295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:47.660766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47590403-8d04-4b2c-bd1b-69589efc7295 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:47.680199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1306/2607] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:48.154266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ebc61-7350-44b1-b4be-e1b57234c61c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:48.156039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-185ebc61-7350-44b1-b4be-e1b57234c61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:48.176554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ebc61-7350-44b1-b4be-e1b57234c61c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:48.184674 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b697c-cc0f-4945-85d0-fca42faf34af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:48.188561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b697c-cc0f-4945-85d0-fca42faf34af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:48.188561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b697c-cc0f-4945-85d0-fca42faf34af admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:48.188561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b697c-cc0f-4945-85d0-fca42faf34af admin admin] PATH_INFO: `/v3/projects/676fb0fe414e4fff9e943caa9f50343a` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:48.191418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b697c-cc0f-4945-85d0-fca42faf34af admin admin] RBAC: Authorizing `identity:delete_project(project_id=676fb0fe414e4fff9e943caa9f50343a)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:48.191706 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b697c-cc0f-4945-85d0-fca42faf34af admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:48.259905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4b4b697c-cc0f-4945-85d0-fca42faf34af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '676fb0fe414e4fff9e943caa9f50343a', 'request_id': 'req-4b4b697c-cc0f-4945-85d0-fca42faf34af'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:48.269891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4b4b697c-cc0f-4945-85d0-fca42faf34af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '676fb0fe414e4fff9e943caa9f50343a', 'request_id': 'req-4b4b697c-cc0f-4945-85d0-fca42faf34af'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:48.277379 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4b4b697c-cc0f-4945-85d0-fca42faf34af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '676fb0fe414e4fff9e943caa9f50343a', 'request_id': 'req-4b4b697c-cc0f-4945-85d0-fca42faf34af'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:48.308651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4b4b697c-cc0f-4945-85d0-fca42faf34af admin admin] The token cache is being invalidate because project 676fb0fe414e4fff9e943caa9f50343a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:25:48.308651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4b4b697c-cc0f-4945-85d0-fca42faf34af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:48.310769 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-4b4b697c-cc0f-4945-85d0-fca42faf34af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:48.312462 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1302/2608] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:25:48 2022] DELETE /identity/v3/projects/676fb0fe414e4fff9e943caa9f50343a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:48.317798 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5c1d9-72aa-4cad-9211-43cf8bf2767e None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:48.318284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5c1d9-72aa-4cad-9211-43cf8bf2767e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:48.318576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5c1d9-72aa-4cad-9211-43cf8bf2767e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:48.346062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-66b5c1d9-72aa-4cad-9211-43cf8bf2767e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:48.348040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-66b5c1d9-72aa-4cad-9211-43cf8bf2767e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:48.436524 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1307/2609] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:48.502394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dc8b9-1b22-430b-98d6-f786f021a11a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:48.502947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dc8b9-1b22-430b-98d6-f786f021a11a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:48.502947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dc8b9-1b22-430b-98d6-f786f021a11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:48.524550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-b33dc8b9-1b22-430b-98d6-f786f021a11a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:48.525801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b33dc8b9-1b22-430b-98d6-f786f021a11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:48.534322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1303/2610] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:48.538606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5c1d9-72aa-4cad-9211-43cf8bf2767e None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:48.549942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5c1d9-72aa-4cad-9211-43cf8bf2767e None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:48.555993 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32951d1-0f15-42b7-a8ce-cf313f063c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:48.556800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32951d1-0f15-42b7-a8ce-cf313f063c22 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:48.556913 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32951d1-0f15-42b7-a8ce-cf313f063c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:48.557159 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32951d1-0f15-42b7-a8ce-cf313f063c22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:48.558458 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32951d1-0f15-42b7-a8ce-cf313f063c22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:25:48.558822 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32951d1-0f15-42b7-a8ce-cf313f063c22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:48.560423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32951d1-0f15-42b7-a8ce-cf313f063c22 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:48.580985 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1308/2611] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:25:48.585446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dc8b9-1b22-430b-98d6-f786f021a11a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:48.596486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dc8b9-1b22-430b-98d6-f786f021a11a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:48.602316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc03f4-a7bc-43c1-b1c6-d9d866e3b634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:48.602945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc03f4-a7bc-43c1-b1c6-d9d866e3b634 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:48.603268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc03f4-a7bc-43c1-b1c6-d9d866e3b634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:48.603546 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc03f4-a7bc-43c1-b1c6-d9d866e3b634 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:48.604208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc03f4-a7bc-43c1-b1c6-d9d866e3b634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:48.606426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc03f4-a7bc-43c1-b1c6-d9d866e3b634 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:48.638572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1304/2612] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:25:48 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:25:48.644180 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32951d1-0f15-42b7-a8ce-cf313f063c22 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:48.659450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32951d1-0f15-42b7-a8ce-cf313f063c22 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:48.728859 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e16155-0169-433e-b035-9127205773f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:48.729612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e16155-0169-433e-b035-9127205773f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:48.730193 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e16155-0169-433e-b035-9127205773f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:48.731138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e16155-0169-433e-b035-9127205773f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:48.731426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e16155-0169-433e-b035-9127205773f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:48.733469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e16155-0169-433e-b035-9127205773f2 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:48.765988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1309/2613] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:25:48 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:48.769977 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc03f4-a7bc-43c1-b1c6-d9d866e3b634 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:48.783333 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc03f4-a7bc-43c1-b1c6-d9d866e3b634 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:48.790265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b616157-bf80-480d-b964-187b3ba0b7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:48.791427 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b616157-bf80-480d-b964-187b3ba0b7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:48.791836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b616157-bf80-480d-b964-187b3ba0b7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:48.792329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b616157-bf80-480d-b964-187b3ba0b7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:48.793192 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b616157-bf80-480d-b964-187b3ba0b7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:48.794974 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b616157-bf80-480d-b964-187b3ba0b7d7 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:48.806968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1305/2614] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:48.811951 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e16155-0169-433e-b035-9127205773f2 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:48.823966 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e16155-0169-433e-b035-9127205773f2 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:48.828858 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d45952-a59d-4d37-935c-b2ceed4c6e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:48.829488 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d45952-a59d-4d37-935c-b2ceed4c6e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:48.831076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d45952-a59d-4d37-935c-b2ceed4c6e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:48.831076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d45952-a59d-4d37-935c-b2ceed4c6e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:48.831076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d45952-a59d-4d37-935c-b2ceed4c6e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:48.832711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d45952-a59d-4d37-935c-b2ceed4c6e17 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:48.848003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1310/2615] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:48.852377 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b616157-bf80-480d-b964-187b3ba0b7d7 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:48.861565 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:48.866920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b616157-bf80-480d-b964-187b3ba0b7d7 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:48.875150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1dc8b-d93c-4c31-9a37-625d3b5d5c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:48.876322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1dc8b-d93c-4c31-9a37-625d3b5d5c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:48.876821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1dc8b-d93c-4c31-9a37-625d3b5d5c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:48.877300 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1dc8b-d93c-4c31-9a37-625d3b5d5c21 admin admin] PATH_INFO: `/v3/projects/6e99d17eda624e1284d6a01e72deddab/users/a61664af3f594375ac41c8aa7423627f/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:48.897404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1dc8b-d93c-4c31-9a37-625d3b5d5c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e99d17eda624e1284d6a01e72deddab, user_id=a61664af3f594375ac41c8aa7423627f, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:48.900500 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1dc8b-d93c-4c31-9a37-625d3b5d5c21 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:48.930165 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1306/2616] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:25:48 2022] PUT /identity/v3/projects/6e99d17eda624e1284d6a01e72deddab/users/a61664af3f594375ac41c8aa7423627f/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:48.942633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a511d-efc8-4101-9df3-d430547a03bf None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:48.942925 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a511d-efc8-4101-9df3-d430547a03bf None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:48.943250 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a511d-efc8-4101-9df3-d430547a03bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:48.979547 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-da1a511d-efc8-4101-9df3-d430547a03bf None None] MFA Rules not processed for user `a61664af3f594375ac41c8aa7423627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:48.980994 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-da1a511d-efc8-4101-9df3-d430547a03bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:49.008236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:49.096609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1311/2617] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:25:49.101853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1dc8b-d93c-4c31-9a37-625d3b5d5c21 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:49.103891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-10a1dc8b-d93c-4c31-9a37-625d3b5d5c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:49.185070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1dc8b-d93c-4c31-9a37-625d3b5d5c21 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:49.190013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f6e67-847f-44bc-8303-76368ff5a744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:49.191381 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f6e67-847f-44bc-8303-76368ff5a744 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:49.191663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f6e67-847f-44bc-8303-76368ff5a744 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:49.191934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f6e67-847f-44bc-8303-76368ff5a744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:49.206445 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f6e67-847f-44bc-8303-76368ff5a744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:49.208310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f6e67-847f-44bc-8303-76368ff5a744 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:49.220375 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1307/2618] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:49.801151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a511d-efc8-4101-9df3-d430547a03bf None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:49.803030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-da1a511d-efc8-4101-9df3-d430547a03bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:49.876975 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a511d-efc8-4101-9df3-d430547a03bf None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:49.884499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50f17c-afbc-4da2-ab60-6d8db9a2c645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:49.885537 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50f17c-afbc-4da2-ab60-6d8db9a2c645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:49.886019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50f17c-afbc-4da2-ab60-6d8db9a2c645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:49.886498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50f17c-afbc-4da2-ab60-6d8db9a2c645 admin admin] PATH_INFO: `/v3/users/b55eed583fac43fb9bfe1366a42bd2fc` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:49.888409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50f17c-afbc-4da2-ab60-6d8db9a2c645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b55eed583fac43fb9bfe1366a42bd2fc)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:49.891953 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50f17c-afbc-4da2-ab60-6d8db9a2c645 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:49.942110 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-be50f17c-afbc-4da2-ab60-6d8db9a2c645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b55eed583fac43fb9bfe1366a42bd2fc'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:49.959842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-be50f17c-afbc-4da2-ab60-6d8db9a2c645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b55eed583fac43fb9bfe1366a42bd2fc'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:49.968088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-be50f17c-afbc-4da2-ab60-6d8db9a2c645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b55eed583fac43fb9bfe1366a42bd2fc'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:49.968932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-be50f17c-afbc-4da2-ab60-6d8db9a2c645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b55eed583fac43fb9bfe1366a42bd2fc'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:49.969659 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-be50f17c-afbc-4da2-ab60-6d8db9a2c645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b55eed583fac43fb9bfe1366a42bd2fc'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:49.987569 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1312/2619] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:25:49 2022] DELETE /identity/v3/users/b55eed583fac43fb9bfe1366a42bd2fc => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:50.175412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f6e67-847f-44bc-8303-76368ff5a744 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:50.188313 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f6e67-847f-44bc-8303-76368ff5a744 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:50.249375 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eae2ec-e8b2-4076-87f8-d8383f3a5624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:50.250032 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eae2ec-e8b2-4076-87f8-d8383f3a5624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:50.250359 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eae2ec-e8b2-4076-87f8-d8383f3a5624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:50.250561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eae2ec-e8b2-4076-87f8-d8383f3a5624 admin admin] PATH_INFO: `/v3/projects/ccb52045ffa14e94998e764c1dbc605e` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:50.251739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eae2ec-e8b2-4076-87f8-d8383f3a5624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccb52045ffa14e94998e764c1dbc605e)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:50.253782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eae2ec-e8b2-4076-87f8-d8383f3a5624 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:50.300982 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-10eae2ec-e8b2-4076-87f8-d8383f3a5624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccb52045ffa14e94998e764c1dbc605e', 'request_id': 'req-10eae2ec-e8b2-4076-87f8-d8383f3a5624'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:50.308326 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-10eae2ec-e8b2-4076-87f8-d8383f3a5624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccb52045ffa14e94998e764c1dbc605e', 'request_id': 'req-10eae2ec-e8b2-4076-87f8-d8383f3a5624'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:50.314033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-10eae2ec-e8b2-4076-87f8-d8383f3a5624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccb52045ffa14e94998e764c1dbc605e', 'request_id': 'req-10eae2ec-e8b2-4076-87f8-d8383f3a5624'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:50.325924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-10eae2ec-e8b2-4076-87f8-d8383f3a5624 admin admin] The token cache is being invalidate because project ccb52045ffa14e94998e764c1dbc605e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:25:50.326211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-10eae2ec-e8b2-4076-87f8-d8383f3a5624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:50.326858 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-10eae2ec-e8b2-4076-87f8-d8383f3a5624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:25:50.328378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1308/2620] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:25:50 2022] DELETE /identity/v3/projects/ccb52045ffa14e94998e764c1dbc605e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:25:50.333383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120aed6e-ee13-41b9-bad7-fb4c5b9c156a None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:50.333661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120aed6e-ee13-41b9-bad7-fb4c5b9c156a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:50.333958 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120aed6e-ee13-41b9-bad7-fb4c5b9c156a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:50.357229 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-120aed6e-ee13-41b9-bad7-fb4c5b9c156a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:50.358622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-120aed6e-ee13-41b9-bad7-fb4c5b9c156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:50.389381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:50.421423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1313/2621] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:50.430600 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02269629-73c0-4252-9ad4-e92c5000ab97 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:50.430888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02269629-73c0-4252-9ad4-e92c5000ab97 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:50.431206 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02269629-73c0-4252-9ad4-e92c5000ab97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:50.451791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-02269629-73c0-4252-9ad4-e92c5000ab97 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:50.453041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-02269629-73c0-4252-9ad4-e92c5000ab97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:50.461166 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1309/2622] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:50.466027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120aed6e-ee13-41b9-bad7-fb4c5b9c156a None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:50.475908 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120aed6e-ee13-41b9-bad7-fb4c5b9c156a None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:50.480791 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bd517-22c2-4cd0-94ad-33dd231f71e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:50.481389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bd517-22c2-4cd0-94ad-33dd231f71e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:50.481675 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bd517-22c2-4cd0-94ad-33dd231f71e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:50.481950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bd517-22c2-4cd0-94ad-33dd231f71e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:50.482605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5bd517-22c2-4cd0-94ad-33dd231f71e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:25:50.482909 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5bd517-22c2-4cd0-94ad-33dd231f71e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:50.484553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5bd517-22c2-4cd0-94ad-33dd231f71e7 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:50.496118 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1314/2623] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:25:50.500677 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02269629-73c0-4252-9ad4-e92c5000ab97 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:50.510114 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02269629-73c0-4252-9ad4-e92c5000ab97 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:50.515421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0567a-6984-4c5c-a236-265e9aa36c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:50.516026 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0567a-6984-4c5c-a236-265e9aa36c64 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:50.516307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0567a-6984-4c5c-a236-265e9aa36c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:50.516578 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0567a-6984-4c5c-a236-265e9aa36c64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:50.517218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0567a-6984-4c5c-a236-265e9aa36c64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:50.519143 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0567a-6984-4c5c-a236-265e9aa36c64 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:50.523670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:50.550115 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1310/2624] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:25:50 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:25:50.554592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bd517-22c2-4cd0-94ad-33dd231f71e7 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:50.565351 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bd517-22c2-4cd0-94ad-33dd231f71e7 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:50.624612 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c716213-ad3e-4090-a379-95c6c71c7a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:50.625226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c716213-ad3e-4090-a379-95c6c71c7a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:50.625510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c716213-ad3e-4090-a379-95c6c71c7a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:50.625783 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c716213-ad3e-4090-a379-95c6c71c7a4f admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:50.626418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c716213-ad3e-4090-a379-95c6c71c7a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:50.628362 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c716213-ad3e-4090-a379-95c6c71c7a4f admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:50.650521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1315/2625] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:25:50 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:25:50.655541 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0567a-6984-4c5c-a236-265e9aa36c64 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:50.667867 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0567a-6984-4c5c-a236-265e9aa36c64 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:50.673252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053c3a8-d065-4a72-8159-b0fc09eed113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:50.674141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053c3a8-d065-4a72-8159-b0fc09eed113 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:50.674261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053c3a8-d065-4a72-8159-b0fc09eed113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:50.674805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053c3a8-d065-4a72-8159-b0fc09eed113 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:50.675405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053c3a8-d065-4a72-8159-b0fc09eed113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:50.677638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053c3a8-d065-4a72-8159-b0fc09eed113 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:50.691777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1311/2626] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:25:50.693051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c716213-ad3e-4090-a379-95c6c71c7a4f admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:50.704346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c716213-ad3e-4090-a379-95c6c71c7a4f admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:50.709846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27092313-51f8-4ff0-8c13-4bc9f14271fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:50.711089 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27092313-51f8-4ff0-8c13-4bc9f14271fd admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:50.711476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27092313-51f8-4ff0-8c13-4bc9f14271fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:50.711752 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27092313-51f8-4ff0-8c13-4bc9f14271fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:50.712414 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27092313-51f8-4ff0-8c13-4bc9f14271fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:50.714138 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27092313-51f8-4ff0-8c13-4bc9f14271fd admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:50.718792 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c23f5-60aa-4b93-8432-3d505a91638f None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:50.719118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c23f5-60aa-4b93-8432-3d505a91638f None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:50.719408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c23f5-60aa-4b93-8432-3d505a91638f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:50.725782 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1316/2627] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:25:50.730258 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27092313-51f8-4ff0-8c13-4bc9f14271fd admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:50.739545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-1d1c23f5-60aa-4b93-8432-3d505a91638f None None] MFA Rules not processed for user `a61664af3f594375ac41c8aa7423627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:50.740753 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1d1c23f5-60aa-4b93-8432-3d505a91638f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:50.741020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27092313-51f8-4ff0-8c13-4bc9f14271fd admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:50.746449 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04349e-9ec5-481c-9960-855397e77a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:50.747227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04349e-9ec5-481c-9960-855397e77a67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:50.747618 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04349e-9ec5-481c-9960-855397e77a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:50.747842 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04349e-9ec5-481c-9960-855397e77a67 admin admin] PATH_INFO: `/v3/projects/9a188483dd484db4962d038428cba54d/users/1686024a93d34902b19c520d2b2a707d/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:50.766019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04349e-9ec5-481c-9960-855397e77a67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a188483dd484db4962d038428cba54d, user_id=1686024a93d34902b19c520d2b2a707d, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:50.768820 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04349e-9ec5-481c-9960-855397e77a67 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:50.784346 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1317/2628] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:25:50 2022] PUT /identity/v3/projects/9a188483dd484db4962d038428cba54d/users/1686024a93d34902b19c520d2b2a707d/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:25:50.790292 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f965b-58df-4313-8cc4-f832df5af529 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:50.790710 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f965b-58df-4313-8cc4-f832df5af529 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:50.791090 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f965b-58df-4313-8cc4-f832df5af529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:50.803435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1312/2629] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:25:50.810822 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-be2f965b-58df-4313-8cc4-f832df5af529 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:50.812070 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-be2f965b-58df-4313-8cc4-f832df5af529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:50.879545 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1318/2630] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:25:50.890534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c23f5-60aa-4b93-8432-3d505a91638f None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:50.892636 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1d1c23f5-60aa-4b93-8432-3d505a91638f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:50.963952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c23f5-60aa-4b93-8432-3d505a91638f None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:51.016291 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365c371-f3b2-44e5-93d6-24a1e65f1de3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:51.016915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365c371-f3b2-44e5-93d6-24a1e65f1de3 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:51.017187 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365c371-f3b2-44e5-93d6-24a1e65f1de3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:51.017451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365c371-f3b2-44e5-93d6-24a1e65f1de3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:51.027783 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365c371-f3b2-44e5-93d6-24a1e65f1de3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:51.029127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365c371-f3b2-44e5-93d6-24a1e65f1de3 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:51.042052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1313/2631] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:55.581075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8d78d-6160-4cc2-8d12-1053a43b7932 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:55.582423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8d78d-6160-4cc2-8d12-1053a43b7932 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:55.582969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8d78d-6160-4cc2-8d12-1053a43b7932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:55.657474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-6df8d78d-6160-4cc2-8d12-1053a43b7932 None None] MFA Rules not processed for user `1686024a93d34902b19c520d2b2a707d`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:55.658981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-6df8d78d-6160-4cc2-8d12-1053a43b7932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:55.676726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:55.835521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1319/2632] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:55.845905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365c371-f3b2-44e5-93d6-24a1e65f1de3 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:55.848888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8365c371-f3b2-44e5-93d6-24a1e65f1de3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:55.922412 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:55.983631 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365c371-f3b2-44e5-93d6-24a1e65f1de3 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:55.993043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4f0ad-d735-47d3-b87a-c015dc5c8887 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:55.994170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4f0ad-d735-47d3-b87a-c015dc5c8887 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:55.994762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4f0ad-d735-47d3-b87a-c015dc5c8887 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:55.997302 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4f0ad-d735-47d3-b87a-c015dc5c8887 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:56.016165 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4f0ad-d735-47d3-b87a-c015dc5c8887 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:56.018176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4f0ad-d735-47d3-b87a-c015dc5c8887 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:56.029635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1314/2633] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:57.937034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da524b-cb14-4a5b-a37d-8b487c70ae5e None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:57.938079 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da524b-cb14-4a5b-a37d-8b487c70ae5e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:57.938830 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da524b-cb14-4a5b-a37d-8b487c70ae5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:57.961695 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-c9da524b-cb14-4a5b-a37d-8b487c70ae5e None None] MFA Rules not processed for user `1686024a93d34902b19c520d2b2a707d`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:57.962908 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c9da524b-cb14-4a5b-a37d-8b487c70ae5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:57.972013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1320/2634] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:25:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:06.024305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=17 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:07.955366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=27 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:14.882094 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd346b78-8382-4239-a248-2060549325b1 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:14.882685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd346b78-8382-4239-a248-2060549325b1 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:14.882685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd346b78-8382-4239-a248-2060549325b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:14.912345 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-dd346b78-8382-4239-a248-2060549325b1 None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:14.914914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-dd346b78-8382-4239-a248-2060549325b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:15.003694 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1315/2635] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:26:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:15.046789 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da524b-cb14-4a5b-a37d-8b487c70ae5e None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:15.048507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c9da524b-cb14-4a5b-a37d-8b487c70ae5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:15.173944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da524b-cb14-4a5b-a37d-8b487c70ae5e None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:15.186005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db8eae-2263-45c4-aafe-ebb46f600b02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:15.186005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db8eae-2263-45c4-aafe-ebb46f600b02 service swift] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:15.186357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db8eae-2263-45c4-aafe-ebb46f600b02 service swift] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:15.186516 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db8eae-2263-45c4-aafe-ebb46f600b02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:15.201810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db8eae-2263-45c4-aafe-ebb46f600b02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:15.204564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db8eae-2263-45c4-aafe-ebb46f600b02 service swift] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:15.216119 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1321/2636] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:26:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:26:16.022580 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0ad61-84aa-4ec2-916a-48e7ca8ebfc7 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:16.023127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0ad61-84aa-4ec2-916a-48e7ca8ebfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:16.023127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0ad61-84aa-4ec2-916a-48e7ca8ebfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:16.053156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-16f0ad61-84aa-4ec2-916a-48e7ca8ebfc7 None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:16.055095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-16f0ad61-84aa-4ec2-916a-48e7ca8ebfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:16.066919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1316/2637] 10.222.0.61 () {58 vars in 967 bytes} [Wed Sep 7 13:26:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:26:16.081604 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db8eae-2263-45c4-aafe-ebb46f600b02 service swift] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:16.097917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db8eae-2263-45c4-aafe-ebb46f600b02 service swift] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:16.105176 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d264fe8-3aaa-4b7b-9415-17e0a3fa151a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:16.107954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d264fe8-3aaa-4b7b-9415-17e0a3fa151a service swift] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:16.108642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d264fe8-3aaa-4b7b-9415-17e0a3fa151a service swift] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:16.109637 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d264fe8-3aaa-4b7b-9415-17e0a3fa151a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:16.119597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d264fe8-3aaa-4b7b-9415-17e0a3fa151a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:16.121609 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d264fe8-3aaa-4b7b-9415-17e0a3fa151a service swift] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:16.131357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1322/2638] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:26:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 13:26:16.437900 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ddd67-a410-41c6-b2bc-d98abb30d347 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:16.438199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ddd67-a410-41c6-b2bc-d98abb30d347 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:16.438441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ddd67-a410-41c6-b2bc-d98abb30d347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:16.459235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-721ddd67-a410-41c6-b2bc-d98abb30d347 None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:16.460587 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-721ddd67-a410-41c6-b2bc-d98abb30d347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:16.468897 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1317/2639] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:26:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:16.482563 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d264fe8-3aaa-4b7b-9415-17e0a3fa151a service swift] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:16.495464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d264fe8-3aaa-4b7b-9415-17e0a3fa151a service swift] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:16.502002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7d14a-8db8-4b5e-9643-22dc91d9b885 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:16.502561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7d14a-8db8-4b5e-9643-22dc91d9b885 service swift] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:16.502847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7d14a-8db8-4b5e-9643-22dc91d9b885 service swift] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:16.503129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7d14a-8db8-4b5e-9643-22dc91d9b885 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:16.523926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7d14a-8db8-4b5e-9643-22dc91d9b885 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:16.527390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7d14a-8db8-4b5e-9643-22dc91d9b885 service swift] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:16.540510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1323/2640] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:26:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:26:17.004493 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79cd95-6bc3-4083-b872-7e885b25bb09 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:17.004775 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79cd95-6bc3-4083-b872-7e885b25bb09 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:17.005048 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79cd95-6bc3-4083-b872-7e885b25bb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:17.032772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-7d79cd95-6bc3-4083-b872-7e885b25bb09 None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:17.034378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7d79cd95-6bc3-4083-b872-7e885b25bb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:17.044874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1318/2641] 10.222.0.61 () {58 vars in 967 bytes} [Wed Sep 7 13:26:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:17.060622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7d14a-8db8-4b5e-9643-22dc91d9b885 service swift] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:17.071558 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7d14a-8db8-4b5e-9643-22dc91d9b885 service swift] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:17.077797 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb704f4-9d88-4d20-aecc-0674d243dae5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:17.078435 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb704f4-9d88-4d20-aecc-0674d243dae5 service swift] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:17.078717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb704f4-9d88-4d20-aecc-0674d243dae5 service swift] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:17.078997 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb704f4-9d88-4d20-aecc-0674d243dae5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:17.088455 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb704f4-9d88-4d20-aecc-0674d243dae5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:17.090127 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb704f4-9d88-4d20-aecc-0674d243dae5 service swift] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:17.100198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1324/2642] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:26:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:26:19.350076 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79cd95-6bc3-4083-b872-7e885b25bb09 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:19.352810 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7d79cd95-6bc3-4083-b872-7e885b25bb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:19.372813 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79cd95-6bc3-4083-b872-7e885b25bb09 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:19.378647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:19.379667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:19.380281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:19.380853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] PATH_INFO: `/v3/users/a61664af3f594375ac41c8aa7423627f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:19.382524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a61664af3f594375ac41c8aa7423627f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:19.384575 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:19.446702 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb704f4-9d88-4d20-aecc-0674d243dae5 service swift] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:19.448348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4eb704f4-9d88-4d20-aecc-0674d243dae5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:19.466566 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a61664af3f594375ac41c8aa7423627f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:19.470402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb704f4-9d88-4d20-aecc-0674d243dae5 service swift] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:19.476027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be6b8e-31e4-4a71-b4f1-a44cb8122be2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:19.476665 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be6b8e-31e4-4a71-b4f1-a44cb8122be2 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:19.476948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be6b8e-31e4-4a71-b4f1-a44cb8122be2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:19.477225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be6b8e-31e4-4a71-b4f1-a44cb8122be2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:19.481058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-90be6b8e-31e4-4a71-b4f1-a44cb8122be2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:19.483454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a61664af3f594375ac41c8aa7423627f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:19.484402 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a61664af3f594375ac41c8aa7423627f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:19.485332 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a61664af3f594375ac41c8aa7423627f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:19.496373 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be6b8e-31e4-4a71-b4f1-a44cb8122be2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:19.497854 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be6b8e-31e4-4a71-b4f1-a44cb8122be2 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:19.505399 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a61664af3f594375ac41c8aa7423627f'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:19.510437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1325/2643] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:26:19.525525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1319/2644] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:26:19 2022] DELETE /identity/v3/users/a61664af3f594375ac41c8aa7423627f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:19.531873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3687-d7da-4b89-93bf-0c47e0693d32 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:19.532181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3687-d7da-4b89-93bf-0c47e0693d32 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:19.532470 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3687-d7da-4b89-93bf-0c47e0693d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:19.557343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-c96e3687-d7da-4b89-93bf-0c47e0693d32 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:19.558617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c96e3687-d7da-4b89-93bf-0c47e0693d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:19.651686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1326/2645] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:19.659821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:19.661383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:19.757881 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4eb49-6380-4f5e-ba5c-14e494cda298 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:19.762503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc35e8-3b09-4bc1-ab6c-c153943f8033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:19.763281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc35e8-3b09-4bc1-ab6c-c153943f8033 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:19.763463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc35e8-3b09-4bc1-ab6c-c153943f8033 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:19.763688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc35e8-3b09-4bc1-ab6c-c153943f8033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:19.774126 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc35e8-3b09-4bc1-ab6c-c153943f8033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:19.776342 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc35e8-3b09-4bc1-ab6c-c153943f8033 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:19.783133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:19.791729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1320/2646] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:20.000574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e3687-d7da-4b89-93bf-0c47e0693d32 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:20.004163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c96e3687-d7da-4b89-93bf-0c47e0693d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:20.027715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e3687-d7da-4b89-93bf-0c47e0693d32 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:20.035569 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5c9a2-3101-49b0-9600-7fa7e91580e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:20.036988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5c9a2-3101-49b0-9600-7fa7e91580e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:20.037483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5c9a2-3101-49b0-9600-7fa7e91580e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:20.037967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5c9a2-3101-49b0-9600-7fa7e91580e0 admin admin] PATH_INFO: `/v3/projects/6e99d17eda624e1284d6a01e72deddab` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:20.040058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad5c9a2-3101-49b0-9600-7fa7e91580e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e99d17eda624e1284d6a01e72deddab)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:20.044932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad5c9a2-3101-49b0-9600-7fa7e91580e0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:20.106272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:20.125641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-aad5c9a2-3101-49b0-9600-7fa7e91580e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e99d17eda624e1284d6a01e72deddab', 'request_id': 'req-aad5c9a2-3101-49b0-9600-7fa7e91580e0'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:20.133448 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-aad5c9a2-3101-49b0-9600-7fa7e91580e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e99d17eda624e1284d6a01e72deddab', 'request_id': 'req-aad5c9a2-3101-49b0-9600-7fa7e91580e0'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:20.142816 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-aad5c9a2-3101-49b0-9600-7fa7e91580e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e99d17eda624e1284d6a01e72deddab', 'request_id': 'req-aad5c9a2-3101-49b0-9600-7fa7e91580e0'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:20.167076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-aad5c9a2-3101-49b0-9600-7fa7e91580e0 admin admin] The token cache is being invalidate because project 6e99d17eda624e1284d6a01e72deddab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:26:20.167511 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-aad5c9a2-3101-49b0-9600-7fa7e91580e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:20.168669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-aad5c9a2-3101-49b0-9600-7fa7e91580e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:20.170692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1327/2647] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:26:19 2022] DELETE /identity/v3/projects/6e99d17eda624e1284d6a01e72deddab => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:23.825831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be5ab8-5dae-4e6b-ad8e-70c7c26d0181 None None] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:23.826714 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be5ab8-5dae-4e6b-ad8e-70c7c26d0181 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:23.826714 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be5ab8-5dae-4e6b-ad8e-70c7c26d0181 None None] PATH_INFO: `/` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:23.828772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1321/2648] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:26:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:26:23.837508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178324ef-eab1-4c2f-b35f-2f7cd57c4c06 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:23.838687 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178324ef-eab1-4c2f-b35f-2f7cd57c4c06 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:23.838828 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178324ef-eab1-4c2f-b35f-2f7cd57c4c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:23.888276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-178324ef-eab1-4c2f-b35f-2f7cd57c4c06 None None] MFA Rules not processed for user `9e283e62741f40439d02e545a212807a`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:23.890395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-178324ef-eab1-4c2f-b35f-2f7cd57c4c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:23.914016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be5ab8-5dae-4e6b-ad8e-70c7c26d0181 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:23.917653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-48be5ab8-5dae-4e6b-ad8e-70c7c26d0181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:23.996925 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1328/2649] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:26:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:24.008381 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178324ef-eab1-4c2f-b35f-2f7cd57c4c06 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:24.009988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-178324ef-eab1-4c2f-b35f-2f7cd57c4c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:24.010791 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be5ab8-5dae-4e6b-ad8e-70c7c26d0181 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:24.017431 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464822d0-88a7-4d03-8392-87f236764c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:24.018564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464822d0-88a7-4d03-8392-87f236764c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:24.019304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464822d0-88a7-4d03-8392-87f236764c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:24.020008 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464822d0-88a7-4d03-8392-87f236764c6e admin admin] PATH_INFO: `/v3/users/181c888fbd454f00b049522fdfb4197e` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:24.026134 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178324ef-eab1-4c2f-b35f-2f7cd57c4c06 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:24.031581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab32917-6c24-41f7-a381-84145531cc6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:24.032245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab32917-6c24-41f7-a381-84145531cc6d service placement] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:24.032530 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab32917-6c24-41f7-a381-84145531cc6d service placement] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:24.032812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab32917-6c24-41f7-a381-84145531cc6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:24.042353 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab32917-6c24-41f7-a381-84145531cc6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:24.042858 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464822d0-88a7-4d03-8392-87f236764c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=181c888fbd454f00b049522fdfb4197e)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:24.043935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab32917-6c24-41f7-a381-84145531cc6d service placement] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:24.045079 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464822d0-88a7-4d03-8392-87f236764c6e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:24.067444 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1329/2650] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:26:24.112831 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-464822d0-88a7-4d03-8392-87f236764c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '181c888fbd454f00b049522fdfb4197e'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:24.121357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-464822d0-88a7-4d03-8392-87f236764c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '181c888fbd454f00b049522fdfb4197e'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:24.122156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-464822d0-88a7-4d03-8392-87f236764c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '181c888fbd454f00b049522fdfb4197e'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:24.122762 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-464822d0-88a7-4d03-8392-87f236764c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '181c888fbd454f00b049522fdfb4197e'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:24.135903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-464822d0-88a7-4d03-8392-87f236764c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '181c888fbd454f00b049522fdfb4197e'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:24.148497 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1322/2651] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:26:23 2022] DELETE /identity/v3/users/181c888fbd454f00b049522fdfb4197e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:26:24.337102 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab32917-6c24-41f7-a381-84145531cc6d service placement] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:24.357234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab32917-6c24-41f7-a381-84145531cc6d service placement] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:24.428019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f76b65-b4bc-41d7-975d-e70fe396954f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:24.428766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f76b65-b4bc-41d7-975d-e70fe396954f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:24.429181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f76b65-b4bc-41d7-975d-e70fe396954f admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:24.429596 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f76b65-b4bc-41d7-975d-e70fe396954f admin admin] PATH_INFO: `/v3/projects/3f1963d0789945fcbd182b7f2ce64d03` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:24.442582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f76b65-b4bc-41d7-975d-e70fe396954f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f1963d0789945fcbd182b7f2ce64d03)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:24.444715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f76b65-b4bc-41d7-975d-e70fe396954f admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:24.470432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:24.496970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-41f76b65-b4bc-41d7-975d-e70fe396954f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f1963d0789945fcbd182b7f2ce64d03', 'request_id': 'req-41f76b65-b4bc-41d7-975d-e70fe396954f'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:24.505075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-41f76b65-b4bc-41d7-975d-e70fe396954f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f1963d0789945fcbd182b7f2ce64d03', 'request_id': 'req-41f76b65-b4bc-41d7-975d-e70fe396954f'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:24.512057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-41f76b65-b4bc-41d7-975d-e70fe396954f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f1963d0789945fcbd182b7f2ce64d03', 'request_id': 'req-41f76b65-b4bc-41d7-975d-e70fe396954f'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:24.527172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-41f76b65-b4bc-41d7-975d-e70fe396954f admin admin] The token cache is being invalidate because project 3f1963d0789945fcbd182b7f2ce64d03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:26:24.527562 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-41f76b65-b4bc-41d7-975d-e70fe396954f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:24.528121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-41f76b65-b4bc-41d7-975d-e70fe396954f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:24.529652 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1330/2652] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:26:24 2022] DELETE /identity/v3/projects/3f1963d0789945fcbd182b7f2ce64d03 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:24.535039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aaf1a-3d3d-41b3-8578-f18e37b901e5 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:24.535355 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aaf1a-3d3d-41b3-8578-f18e37b901e5 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:24.535674 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aaf1a-3d3d-41b3-8578-f18e37b901e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:24.562553 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-5b1aaf1a-3d3d-41b3-8578-f18e37b901e5 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:24.563837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5b1aaf1a-3d3d-41b3-8578-f18e37b901e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:24.639902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1323/2653] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:24.648205 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d962af-5c48-4953-9715-e342e3e56c98 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:24.648487 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d962af-5c48-4953-9715-e342e3e56c98 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:24.648772 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d962af-5c48-4953-9715-e342e3e56c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:24.675186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-57d962af-5c48-4953-9715-e342e3e56c98 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:24.677366 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-57d962af-5c48-4953-9715-e342e3e56c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:24.690835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1331/2654] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:24.696113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1aaf1a-3d3d-41b3-8578-f18e37b901e5 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:24.709095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1aaf1a-3d3d-41b3-8578-f18e37b901e5 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:24.716051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274ed90-840c-47eb-9f94-45ea438104b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:24.716051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274ed90-840c-47eb-9f94-45ea438104b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:24.716463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274ed90-840c-47eb-9f94-45ea438104b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:24.716463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274ed90-840c-47eb-9f94-45ea438104b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:24.717133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274ed90-840c-47eb-9f94-45ea438104b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:26:24.717464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274ed90-840c-47eb-9f94-45ea438104b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:24.719088 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274ed90-840c-47eb-9f94-45ea438104b1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:24.722994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:24.740372 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1324/2655] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:26:24.746118 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d962af-5c48-4953-9715-e342e3e56c98 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:24.758117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d962af-5c48-4953-9715-e342e3e56c98 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:24.763159 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8e1f0-24de-4344-94b8-24d3c0ae388c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:24.763786 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8e1f0-24de-4344-94b8-24d3c0ae388c admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:24.764108 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8e1f0-24de-4344-94b8-24d3c0ae388c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:24.764326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8e1f0-24de-4344-94b8-24d3c0ae388c admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:24.764965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8e1f0-24de-4344-94b8-24d3c0ae388c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:24.766903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8e1f0-24de-4344-94b8-24d3c0ae388c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:24.803793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1332/2656] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:26:24 2022] POST /identity/v3/projects => generated 405 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:24.810811 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274ed90-840c-47eb-9f94-45ea438104b1 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:24.823171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274ed90-840c-47eb-9f94-45ea438104b1 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:24.887998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec7387-139a-45d3-b6e0-e56bf1c86234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:24.889492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec7387-139a-45d3-b6e0-e56bf1c86234 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:24.889492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec7387-139a-45d3-b6e0-e56bf1c86234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:24.889758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec7387-139a-45d3-b6e0-e56bf1c86234 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:24.890624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec7387-139a-45d3-b6e0-e56bf1c86234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:24.892677 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec7387-139a-45d3-b6e0-e56bf1c86234 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:24.920997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1325/2657] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:26:24 2022] POST /identity/v3/users => generated 303 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:26:24.926006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8e1f0-24de-4344-94b8-24d3c0ae388c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:24.941943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8e1f0-24de-4344-94b8-24d3c0ae388c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:24.951922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22529d18-0dad-4210-84d7-14a945235208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:24.952991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22529d18-0dad-4210-84d7-14a945235208 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:24.953494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22529d18-0dad-4210-84d7-14a945235208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:24.954005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22529d18-0dad-4210-84d7-14a945235208 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:24.955294 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22529d18-0dad-4210-84d7-14a945235208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:24.957709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22529d18-0dad-4210-84d7-14a945235208 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:24.970073 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1333/2658] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:24.974505 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec7387-139a-45d3-b6e0-e56bf1c86234 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:24.985754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec7387-139a-45d3-b6e0-e56bf1c86234 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:24.990788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfe0ca-ddd7-4d0e-acf6-5da8bf0b11b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:24.991455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfe0ca-ddd7-4d0e-acf6-5da8bf0b11b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:24.991760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfe0ca-ddd7-4d0e-acf6-5da8bf0b11b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:24.992036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfe0ca-ddd7-4d0e-acf6-5da8bf0b11b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:24.992703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfe0ca-ddd7-4d0e-acf6-5da8bf0b11b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:24.994352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfe0ca-ddd7-4d0e-acf6-5da8bf0b11b1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:25.006341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1326/2659] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:25.011820 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22529d18-0dad-4210-84d7-14a945235208 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:25.023589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22529d18-0dad-4210-84d7-14a945235208 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:25.028942 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce75287-e9c8-41ce-8314-ff5269e9afc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:25.029616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce75287-e9c8-41ce-8314-ff5269e9afc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:25.029882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce75287-e9c8-41ce-8314-ff5269e9afc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:25.030155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce75287-e9c8-41ce-8314-ff5269e9afc6 admin admin] PATH_INFO: `/v3/projects/fbf5db675e0c444e91c5b0b0e0e66371/users/5daffa697d6e49728181354e82430f2c/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:25.050324 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce75287-e9c8-41ce-8314-ff5269e9afc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf5db675e0c444e91c5b0b0e0e66371, user_id=5daffa697d6e49728181354e82430f2c, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:25.053428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce75287-e9c8-41ce-8314-ff5269e9afc6 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:25.073396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1334/2660] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:25 2022] PUT /identity/v3/projects/fbf5db675e0c444e91c5b0b0e0e66371/users/5daffa697d6e49728181354e82430f2c/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:26:25.090730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e7b52-a12b-4101-99bd-f9215ced3838 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:25.090730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e7b52-a12b-4101-99bd-f9215ced3838 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:25.091144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e7b52-a12b-4101-99bd-f9215ced3838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:25.123653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-e78e7b52-a12b-4101-99bd-f9215ced3838 None None] MFA Rules not processed for user `5daffa697d6e49728181354e82430f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:25.125380 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e78e7b52-a12b-4101-99bd-f9215ced3838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:25.219559 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1327/2661] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:25 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:26:25.226029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce75287-e9c8-41ce-8314-ff5269e9afc6 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:25.304312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce75287-e9c8-41ce-8314-ff5269e9afc6 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:25.311557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8cbe7-2326-4a3d-a1cc-afa508ff362a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:25.312355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8cbe7-2326-4a3d-a1cc-afa508ff362a admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:25.312715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8cbe7-2326-4a3d-a1cc-afa508ff362a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:25.313027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8cbe7-2326-4a3d-a1cc-afa508ff362a admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:25.314023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8cbe7-2326-4a3d-a1cc-afa508ff362a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:25.316978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8cbe7-2326-4a3d-a1cc-afa508ff362a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:25.352970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1335/2662] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:26:25 2022] POST /identity/v3/projects => generated 405 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:26:25.356879 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e7b52-a12b-4101-99bd-f9215ced3838 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:25.368640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e7b52-a12b-4101-99bd-f9215ced3838 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:25.452859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df37bc-c654-4bee-adde-18cbd12d709f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:25.453541 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df37bc-c654-4bee-adde-18cbd12d709f admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:25.453875 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df37bc-c654-4bee-adde-18cbd12d709f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:25.454189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df37bc-c654-4bee-adde-18cbd12d709f admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:25.454997 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df37bc-c654-4bee-adde-18cbd12d709f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:25.457059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df37bc-c654-4bee-adde-18cbd12d709f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:25.468275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:25.487579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1328/2663] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:26:25 2022] POST /identity/v3/users => generated 309 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:25.492610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8cbe7-2326-4a3d-a1cc-afa508ff362a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:25.505908 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8cbe7-2326-4a3d-a1cc-afa508ff362a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:25.512195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2efb1-64fb-49fa-bd0b-b33d50eba301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:25.512835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2efb1-64fb-49fa-bd0b-b33d50eba301 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:25.513117 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2efb1-64fb-49fa-bd0b-b33d50eba301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:25.513391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2efb1-64fb-49fa-bd0b-b33d50eba301 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:25.514135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2efb1-64fb-49fa-bd0b-b33d50eba301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:25.515760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2efb1-64fb-49fa-bd0b-b33d50eba301 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:25.531710 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1336/2664] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:26:25.536673 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df37bc-c654-4bee-adde-18cbd12d709f admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:25.548917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df37bc-c654-4bee-adde-18cbd12d709f admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:25.553862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed81e9-897a-4cb3-8c68-724e2af44a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:25.554539 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed81e9-897a-4cb3-8c68-724e2af44a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:25.554819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed81e9-897a-4cb3-8c68-724e2af44a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:25.555093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed81e9-897a-4cb3-8c68-724e2af44a92 admin admin] PATH_INFO: `/v3/projects/df48ca85f6f4407fa8909d24b822df26/users/cecd53b984e14adcb4224e5930401d9e/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:25.572581 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed81e9-897a-4cb3-8c68-724e2af44a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df48ca85f6f4407fa8909d24b822df26, user_id=cecd53b984e14adcb4224e5930401d9e, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:25.576550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed81e9-897a-4cb3-8c68-724e2af44a92 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:25.592833 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1329/2665] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:25 2022] PUT /identity/v3/projects/df48ca85f6f4407fa8909d24b822df26/users/cecd53b984e14adcb4224e5930401d9e/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:25.597826 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2efb1-64fb-49fa-bd0b-b33d50eba301 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:25.616194 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2efb1-64fb-49fa-bd0b-b33d50eba301 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:25.637794 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:25.686717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293b127-068f-4937-90fb-ecd4d6dfca57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:25.687375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293b127-068f-4937-90fb-ecd4d6dfca57 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:25.687650 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293b127-068f-4937-90fb-ecd4d6dfca57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:25.687921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293b127-068f-4937-90fb-ecd4d6dfca57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:25.688564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293b127-068f-4937-90fb-ecd4d6dfca57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:25.690159 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293b127-068f-4937-90fb-ecd4d6dfca57 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:25.700974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1337/2666] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:25.706303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed81e9-897a-4cb3-8c68-724e2af44a92 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:25.718052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed81e9-897a-4cb3-8c68-724e2af44a92 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:25.725673 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bebcf8-6cea-4f75-a3df-c7e614aa9bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:25.726357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bebcf8-6cea-4f75-a3df-c7e614aa9bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:25.726628 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bebcf8-6cea-4f75-a3df-c7e614aa9bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:25.726932 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bebcf8-6cea-4f75-a3df-c7e614aa9bf7 admin admin] PATH_INFO: `/v3/projects/df48ca85f6f4407fa8909d24b822df26/users/cecd53b984e14adcb4224e5930401d9e/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:25.729660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bebcf8-6cea-4f75-a3df-c7e614aa9bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df48ca85f6f4407fa8909d24b822df26, user_id=cecd53b984e14adcb4224e5930401d9e, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:25.732621 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bebcf8-6cea-4f75-a3df-c7e614aa9bf7 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:25.743949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1330/2667] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:25 2022] PUT /identity/v3/projects/df48ca85f6f4407fa8909d24b822df26/users/cecd53b984e14adcb4224e5930401d9e/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:26:25.755668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f1cfd-50f5-4a5b-af7d-dd7125f64bcc None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:25.755957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f1cfd-50f5-4a5b-af7d-dd7125f64bcc None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:25.756240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f1cfd-50f5-4a5b-af7d-dd7125f64bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:25.788788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-d96f1cfd-50f5-4a5b-af7d-dd7125f64bcc None None] MFA Rules not processed for user `cecd53b984e14adcb4224e5930401d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:25.790015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d96f1cfd-50f5-4a5b-af7d-dd7125f64bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:25.888506 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1338/2668] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:26:25.900380 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bebcf8-6cea-4f75-a3df-c7e614aa9bf7 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:25.902589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d7bebcf8-6cea-4f75-a3df-c7e614aa9bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:25.965102 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bebcf8-6cea-4f75-a3df-c7e614aa9bf7 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:25.969068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494bf6e-41e7-463f-a0c3-8ed11ab0ca75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:25.969678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494bf6e-41e7-463f-a0c3-8ed11ab0ca75 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:25.970015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494bf6e-41e7-463f-a0c3-8ed11ab0ca75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:25.970293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494bf6e-41e7-463f-a0c3-8ed11ab0ca75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:25.981175 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494bf6e-41e7-463f-a0c3-8ed11ab0ca75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:25.982636 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494bf6e-41e7-463f-a0c3-8ed11ab0ca75 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:25.993096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1331/2669] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:25 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:26.165848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f1cfd-50f5-4a5b-af7d-dd7125f64bcc None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:26.250957 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f1cfd-50f5-4a5b-af7d-dd7125f64bcc None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:26.257975 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:26.258617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:26.258894 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:26.259204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] PATH_INFO: `/v3/users/5daffa697d6e49728181354e82430f2c` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:26.260805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5daffa697d6e49728181354e82430f2c)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:26.264549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:26.330884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:26.333029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5daffa697d6e49728181354e82430f2c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:26.348136 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5daffa697d6e49728181354e82430f2c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:26.356349 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5daffa697d6e49728181354e82430f2c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:26.357126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5daffa697d6e49728181354e82430f2c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:26.357825 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5daffa697d6e49728181354e82430f2c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:26.374470 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1339/2670] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:26:26 2022] DELETE /identity/v3/users/5daffa697d6e49728181354e82430f2c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:26.379167 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf0e5a-9995-455e-bf2b-7512a24cd6ed None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:26.379454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf0e5a-9995-455e-bf2b-7512a24cd6ed None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:26.379729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf0e5a-9995-455e-bf2b-7512a24cd6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:26.401619 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-1ebf0e5a-9995-455e-bf2b-7512a24cd6ed None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:26.402736 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1ebf0e5a-9995-455e-bf2b-7512a24cd6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:26.479919 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1332/2671] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:26.490319 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:26.491929 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:26.567934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed31c38-e6a5-4f15-a300-16c02316b54c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:26.572206 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75cb1e-2e61-48b3-a2e7-1a48050c84d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:26.573483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75cb1e-2e61-48b3-a2e7-1a48050c84d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:26.573828 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75cb1e-2e61-48b3-a2e7-1a48050c84d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:26.574180 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75cb1e-2e61-48b3-a2e7-1a48050c84d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:26.590272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75cb1e-2e61-48b3-a2e7-1a48050c84d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:26.591971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75cb1e-2e61-48b3-a2e7-1a48050c84d7 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:26.603243 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1340/2672] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:27.158992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf0e5a-9995-455e-bf2b-7512a24cd6ed None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:27.160451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1ebf0e5a-9995-455e-bf2b-7512a24cd6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:27.176212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf0e5a-9995-455e-bf2b-7512a24cd6ed None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:27.182556 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:27.183267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:27.183503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:27.183803 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] PATH_INFO: `/v3/projects/fbf5db675e0c444e91c5b0b0e0e66371` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:27.184971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf5db675e0c444e91c5b0b0e0e66371)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:27.187091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:27.239089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf5db675e0c444e91c5b0b0e0e66371', 'request_id': 'req-019e4f60-e17a-47fd-beea-471d4e9024f3'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.244124 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:27.251723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf5db675e0c444e91c5b0b0e0e66371', 'request_id': 'req-019e4f60-e17a-47fd-beea-471d4e9024f3'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.260437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf5db675e0c444e91c5b0b0e0e66371', 'request_id': 'req-019e4f60-e17a-47fd-beea-471d4e9024f3'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.279057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] The token cache is being invalidate because project fbf5db675e0c444e91c5b0b0e0e66371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:26:27.279541 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.280488 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.282723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1333/2673] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:26:27 2022] DELETE /identity/v3/projects/fbf5db675e0c444e91c5b0b0e0e66371 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:27.287906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75cb1e-2e61-48b3-a2e7-1a48050c84d7 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:27.299409 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75cb1e-2e61-48b3-a2e7-1a48050c84d7 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:27.367391 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808edab1-4e5b-4056-a3d9-a948d3d66f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:27.368002 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808edab1-4e5b-4056-a3d9-a948d3d66f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:27.368261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808edab1-4e5b-4056-a3d9-a948d3d66f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:27.368579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808edab1-4e5b-4056-a3d9-a948d3d66f8d admin admin] PATH_INFO: `/v3/users/cecd53b984e14adcb4224e5930401d9e` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:27.370224 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808edab1-4e5b-4056-a3d9-a948d3d66f8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cecd53b984e14adcb4224e5930401d9e)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:27.372169 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808edab1-4e5b-4056-a3d9-a948d3d66f8d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:27.425873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-808edab1-4e5b-4056-a3d9-a948d3d66f8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cecd53b984e14adcb4224e5930401d9e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.442549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-808edab1-4e5b-4056-a3d9-a948d3d66f8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cecd53b984e14adcb4224e5930401d9e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.446915 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:27.455301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-808edab1-4e5b-4056-a3d9-a948d3d66f8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cecd53b984e14adcb4224e5930401d9e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.456748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-808edab1-4e5b-4056-a3d9-a948d3d66f8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cecd53b984e14adcb4224e5930401d9e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.457682 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-808edab1-4e5b-4056-a3d9-a948d3d66f8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cecd53b984e14adcb4224e5930401d9e'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.477585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1341/2674] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:26:27 2022] DELETE /identity/v3/users/cecd53b984e14adcb4224e5930401d9e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:27.793358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:27.794929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:27.812868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e4f60-e17a-47fd-beea-471d4e9024f3 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:27.887623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57500235-e96b-4da3-8d52-13ebf51e6078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:27.888424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57500235-e96b-4da3-8d52-13ebf51e6078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:27.888788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57500235-e96b-4da3-8d52-13ebf51e6078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:27.889062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57500235-e96b-4da3-8d52-13ebf51e6078 admin admin] PATH_INFO: `/v3/projects/df48ca85f6f4407fa8909d24b822df26` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:27.890662 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57500235-e96b-4da3-8d52-13ebf51e6078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df48ca85f6f4407fa8909d24b822df26)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:27.892903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57500235-e96b-4da3-8d52-13ebf51e6078 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:27.949306 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-57500235-e96b-4da3-8d52-13ebf51e6078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df48ca85f6f4407fa8909d24b822df26', 'request_id': 'req-57500235-e96b-4da3-8d52-13ebf51e6078'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.958203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-57500235-e96b-4da3-8d52-13ebf51e6078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df48ca85f6f4407fa8909d24b822df26', 'request_id': 'req-57500235-e96b-4da3-8d52-13ebf51e6078'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.964939 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-57500235-e96b-4da3-8d52-13ebf51e6078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df48ca85f6f4407fa8909d24b822df26', 'request_id': 'req-57500235-e96b-4da3-8d52-13ebf51e6078'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.981533 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-57500235-e96b-4da3-8d52-13ebf51e6078 admin admin] The token cache is being invalidate because project df48ca85f6f4407fa8909d24b822df26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:26:27.981823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-57500235-e96b-4da3-8d52-13ebf51e6078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.982483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-57500235-e96b-4da3-8d52-13ebf51e6078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:27.984440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1334/2675] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:26:27 2022] DELETE /identity/v3/projects/df48ca85f6f4407fa8909d24b822df26 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:27.989807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3aa0a-2897-49f5-8cce-83f002e902bf None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:27.990091 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3aa0a-2897-49f5-8cce-83f002e902bf None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:27.990455 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3aa0a-2897-49f5-8cce-83f002e902bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.013981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-40f3aa0a-2897-49f5-8cce-83f002e902bf None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:28.017523 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-40f3aa0a-2897-49f5-8cce-83f002e902bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:28.102802 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1342/2676] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:26:28.111433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41bdb2-389e-4437-a0ea-ac7d7cc525b2 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:28.111710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41bdb2-389e-4437-a0ea-ac7d7cc525b2 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:28.111988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41bdb2-389e-4437-a0ea-ac7d7cc525b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.134764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ee41bdb2-389e-4437-a0ea-ac7d7cc525b2 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:28.136637 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ee41bdb2-389e-4437-a0ea-ac7d7cc525b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:28.145140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1335/2677] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:28.150025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3aa0a-2897-49f5-8cce-83f002e902bf None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:28.161352 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3aa0a-2897-49f5-8cce-83f002e902bf None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:28.166960 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a3e5b-8e5e-4462-bb65-d0744f850b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:28.167727 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a3e5b-8e5e-4462-bb65-d0744f850b4a admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:28.168212 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a3e5b-8e5e-4462-bb65-d0744f850b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:28.168605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a3e5b-8e5e-4462-bb65-d0744f850b4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.169266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a3e5b-8e5e-4462-bb65-d0744f850b4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:26:28.169575 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a3e5b-8e5e-4462-bb65-d0744f850b4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:28.171230 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a3e5b-8e5e-4462-bb65-d0744f850b4a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:28.185834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1343/2678] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:26:28.190318 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41bdb2-389e-4437-a0ea-ac7d7cc525b2 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:28.202111 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41bdb2-389e-4437-a0ea-ac7d7cc525b2 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:28.207424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502156f3-889b-413d-8f3d-21f4e572e142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:28.208051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502156f3-889b-413d-8f3d-21f4e572e142 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:28.208360 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502156f3-889b-413d-8f3d-21f4e572e142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:28.208632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502156f3-889b-413d-8f3d-21f4e572e142 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.209281 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502156f3-889b-413d-8f3d-21f4e572e142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:28.211228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502156f3-889b-413d-8f3d-21f4e572e142 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:28.242246 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1336/2679] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:26:28 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:26:28.247267 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a3e5b-8e5e-4462-bb65-d0744f850b4a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:28.262451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a3e5b-8e5e-4462-bb65-d0744f850b4a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:28.325097 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558a77c-6560-4518-909d-fd38e2899d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:28.325788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558a77c-6560-4518-909d-fd38e2899d18 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:28.325788 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558a77c-6560-4518-909d-fd38e2899d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:28.325979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558a77c-6560-4518-909d-fd38e2899d18 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.326615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558a77c-6560-4518-909d-fd38e2899d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:28.330510 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558a77c-6560-4518-909d-fd38e2899d18 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:28.356971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1344/2680] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:26:28 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:28.364093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502156f3-889b-413d-8f3d-21f4e572e142 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:28.379398 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502156f3-889b-413d-8f3d-21f4e572e142 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:28.384173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b3495-5396-45ba-8b67-88a58fbd1ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:28.384790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b3495-5396-45ba-8b67-88a58fbd1ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:28.385112 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b3495-5396-45ba-8b67-88a58fbd1ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:28.385934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b3495-5396-45ba-8b67-88a58fbd1ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.386588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b3495-5396-45ba-8b67-88a58fbd1ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:28.388328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b3495-5396-45ba-8b67-88a58fbd1ccf admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:28.399190 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1337/2681] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:28.404320 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558a77c-6560-4518-909d-fd38e2899d18 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:28.425806 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558a77c-6560-4518-909d-fd38e2899d18 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:28.430824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a6fab-e525-4f42-bb1d-3b90c5abac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:28.431494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a6fab-e525-4f42-bb1d-3b90c5abac66 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:28.432434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a6fab-e525-4f42-bb1d-3b90c5abac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:28.432560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a6fab-e525-4f42-bb1d-3b90c5abac66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.433281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a6fab-e525-4f42-bb1d-3b90c5abac66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:28.434856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a6fab-e525-4f42-bb1d-3b90c5abac66 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:28.446808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1345/2682] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:26:28.452011 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b3495-5396-45ba-8b67-88a58fbd1ccf admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:28.462902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b3495-5396-45ba-8b67-88a58fbd1ccf admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:28.468383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a202d-44a8-41ae-9f35-639014f34b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:28.469073 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a202d-44a8-41ae-9f35-639014f34b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:28.469357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a202d-44a8-41ae-9f35-639014f34b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:28.469635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a202d-44a8-41ae-9f35-639014f34b2a admin admin] PATH_INFO: `/v3/projects/21178cd5f6c745809afb68bc8f7dc38b/users/d1194d210dd6456486437b45346fc799/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.494715 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a202d-44a8-41ae-9f35-639014f34b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21178cd5f6c745809afb68bc8f7dc38b, user_id=d1194d210dd6456486437b45346fc799, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:28.497643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a202d-44a8-41ae-9f35-639014f34b2a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:28.515886 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1338/2683] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:28 2022] PUT /identity/v3/projects/21178cd5f6c745809afb68bc8f7dc38b/users/d1194d210dd6456486437b45346fc799/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:28.528699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca2a4e-5af3-494d-b9d1-ac0ce882ca77 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:28.528988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca2a4e-5af3-494d-b9d1-ac0ce882ca77 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:28.529276 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca2a4e-5af3-494d-b9d1-ac0ce882ca77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.570963 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-6fca2a4e-5af3-494d-b9d1-ac0ce882ca77 None None] MFA Rules not processed for user `d1194d210dd6456486437b45346fc799`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:28.572259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-6fca2a4e-5af3-494d-b9d1-ac0ce882ca77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:28.620158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:28.661076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1346/2684] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:28.665934 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a202d-44a8-41ae-9f35-639014f34b2a admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:28.677850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:28.750165 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a202d-44a8-41ae-9f35-639014f34b2a admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:28.755738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ce333-d176-406b-9de9-c98e4a66ed32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:28.756504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ce333-d176-406b-9de9-c98e4a66ed32 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:28.756790 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ce333-d176-406b-9de9-c98e4a66ed32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:28.757065 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ce333-d176-406b-9de9-c98e4a66ed32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.757741 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ce333-d176-406b-9de9-c98e4a66ed32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:28.759763 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ce333-d176-406b-9de9-c98e4a66ed32 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:28.790935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1339/2685] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:26:28 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:26:28.795903 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca2a4e-5af3-494d-b9d1-ac0ce882ca77 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:28.807797 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca2a4e-5af3-494d-b9d1-ac0ce882ca77 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:28.876414 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b712e-65f8-4895-b222-e036bd54b08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:28.877153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b712e-65f8-4895-b222-e036bd54b08c admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:28.877527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b712e-65f8-4895-b222-e036bd54b08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:28.877916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b712e-65f8-4895-b222-e036bd54b08c admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.878635 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b712e-65f8-4895-b222-e036bd54b08c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:28.880760 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b712e-65f8-4895-b222-e036bd54b08c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:28.906847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1347/2686] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:26:28 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:28.911637 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ce333-d176-406b-9de9-c98e4a66ed32 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:28.925721 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ce333-d176-406b-9de9-c98e4a66ed32 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:28.933632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0d8b8-bfaa-425d-b153-ba28ef957d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:28.934429 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0d8b8-bfaa-425d-b153-ba28ef957d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:28.934842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0d8b8-bfaa-425d-b153-ba28ef957d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:28.935148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0d8b8-bfaa-425d-b153-ba28ef957d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.935818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0d8b8-bfaa-425d-b153-ba28ef957d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:28.937484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0d8b8-bfaa-425d-b153-ba28ef957d62 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:28.949070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1340/2687] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:26:28.956636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b712e-65f8-4895-b222-e036bd54b08c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:28.970093 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b712e-65f8-4895-b222-e036bd54b08c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:28.975716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c4390-1b48-4e5f-991d-7a223967f28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:28.976429 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c4390-1b48-4e5f-991d-7a223967f28d admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:28.976809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c4390-1b48-4e5f-991d-7a223967f28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:28.977031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c4390-1b48-4e5f-991d-7a223967f28d admin admin] PATH_INFO: `/v3/projects/c7902211466f4e17b0b549fed4cee5d0/users/841e5fd12bac450fb7bbcee30fe7e937/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.995417 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c4390-1b48-4e5f-991d-7a223967f28d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7902211466f4e17b0b549fed4cee5d0, user_id=841e5fd12bac450fb7bbcee30fe7e937, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:28.998137 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c4390-1b48-4e5f-991d-7a223967f28d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:29.016083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1348/2688] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:28 2022] PUT /identity/v3/projects/c7902211466f4e17b0b549fed4cee5d0/users/841e5fd12bac450fb7bbcee30fe7e937/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:26:29.022474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0d8b8-bfaa-425d-b153-ba28ef957d62 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:29.039214 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0d8b8-bfaa-425d-b153-ba28ef957d62 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:29.143420 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ddd29-9d80-4377-b956-60aeae792d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:29.144573 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ddd29-9d80-4377-b956-60aeae792d7f admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:29.145072 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ddd29-9d80-4377-b956-60aeae792d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:29.145814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ddd29-9d80-4377-b956-60aeae792d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:29.147042 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ddd29-9d80-4377-b956-60aeae792d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:29.150476 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ddd29-9d80-4377-b956-60aeae792d7f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:29.171212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1341/2689] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:29.176365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c4390-1b48-4e5f-991d-7a223967f28d admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:29.193184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c4390-1b48-4e5f-991d-7a223967f28d admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:29.202178 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1a8b7-33d1-4595-a32c-08bfe73885c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:29.203843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1a8b7-33d1-4595-a32c-08bfe73885c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:29.204066 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1a8b7-33d1-4595-a32c-08bfe73885c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:29.204466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1a8b7-33d1-4595-a32c-08bfe73885c4 admin admin] PATH_INFO: `/v3/projects/c7902211466f4e17b0b549fed4cee5d0/users/841e5fd12bac450fb7bbcee30fe7e937/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:29.208519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee1a8b7-33d1-4595-a32c-08bfe73885c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7902211466f4e17b0b549fed4cee5d0, user_id=841e5fd12bac450fb7bbcee30fe7e937, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:29.214073 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee1a8b7-33d1-4595-a32c-08bfe73885c4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:29.229020 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1349/2690] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:29 2022] PUT /identity/v3/projects/c7902211466f4e17b0b549fed4cee5d0/users/841e5fd12bac450fb7bbcee30fe7e937/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:26:29.245348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68820e5-564c-4c69-8ea8-63c7cd99a52a None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:29.245950 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68820e5-564c-4c69-8ea8-63c7cd99a52a None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:29.246310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68820e5-564c-4c69-8ea8-63c7cd99a52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:29.290303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-d68820e5-564c-4c69-8ea8-63c7cd99a52a None None] MFA Rules not processed for user `841e5fd12bac450fb7bbcee30fe7e937`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:29.291748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d68820e5-564c-4c69-8ea8-63c7cd99a52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:29.375123 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1342/2691] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:29 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:29.392350 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1a8b7-33d1-4595-a32c-08bfe73885c4 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:29.395390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5ee1a8b7-33d1-4595-a32c-08bfe73885c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:29.481941 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1a8b7-33d1-4595-a32c-08bfe73885c4 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:29.486348 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c7646-ca78-4091-954e-17d8a30569fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:29.487005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c7646-ca78-4091-954e-17d8a30569fe service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:29.487442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c7646-ca78-4091-954e-17d8a30569fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:29.487926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c7646-ca78-4091-954e-17d8a30569fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:29.502340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c7646-ca78-4091-954e-17d8a30569fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:29.504295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c7646-ca78-4091-954e-17d8a30569fe service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:29.515234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1350/2692] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:26:29.569907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68820e5-564c-4c69-8ea8-63c7cd99a52a None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:29.571703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d68820e5-564c-4c69-8ea8-63c7cd99a52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:29.584207 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:29.591085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68820e5-564c-4c69-8ea8-63c7cd99a52a None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:29.597491 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048c77e-48c9-4350-8c10-fd95fac097fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:29.598135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048c77e-48c9-4350-8c10-fd95fac097fd service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:29.598484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048c77e-48c9-4350-8c10-fd95fac097fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:29.598756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048c77e-48c9-4350-8c10-fd95fac097fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:29.673301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048c77e-48c9-4350-8c10-fd95fac097fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:29.676129 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048c77e-48c9-4350-8c10-fd95fac097fd service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:29.689503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1343/2693] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:26:33.424562 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c7646-ca78-4091-954e-17d8a30569fe service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:33.470245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:33.506740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c7646-ca78-4091-954e-17d8a30569fe service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:33.514341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:33.515505 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:33.516049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:33.516560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] PATH_INFO: `/v3/users/d1194d210dd6456486437b45346fc799` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:33.518469 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1194d210dd6456486437b45346fc799)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:33.522058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:33.600766 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1194d210dd6456486437b45346fc799'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:33.615315 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1194d210dd6456486437b45346fc799'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:33.624428 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1194d210dd6456486437b45346fc799'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:33.625983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1194d210dd6456486437b45346fc799'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:33.626687 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1194d210dd6456486437b45346fc799'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:33.642991 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1351/2694] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:26:33 2022] DELETE /identity/v3/users/d1194d210dd6456486437b45346fc799 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:33.648719 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922bd1b-ea0a-41c7-89fb-92b041cf1831 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:33.649235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922bd1b-ea0a-41c7-89fb-92b041cf1831 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:33.649361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922bd1b-ea0a-41c7-89fb-92b041cf1831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:33.670877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-b922bd1b-ea0a-41c7-89fb-92b041cf1831 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:33.672180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b922bd1b-ea0a-41c7-89fb-92b041cf1831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:33.745657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1344/2695] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:26:33.756247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:33.757813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:33.821620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953824c-916b-400c-b00d-9de41cc1f14a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:33.826267 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d74a2-5577-494e-8b5e-06dd93977152 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:33.826958 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d74a2-5577-494e-8b5e-06dd93977152 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:33.827397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d74a2-5577-494e-8b5e-06dd93977152 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:33.827783 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d74a2-5577-494e-8b5e-06dd93977152 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:33.838901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d74a2-5577-494e-8b5e-06dd93977152 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:33.840934 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d74a2-5577-494e-8b5e-06dd93977152 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:33.852418 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1352/2696] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:26:34.164621 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922bd1b-ea0a-41c7-89fb-92b041cf1831 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:34.166474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b922bd1b-ea0a-41c7-89fb-92b041cf1831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:34.180058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922bd1b-ea0a-41c7-89fb-92b041cf1831 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:34.186802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:34.187815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:34.187815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:34.187815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] PATH_INFO: `/v3/projects/21178cd5f6c745809afb68bc8f7dc38b` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:34.189239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21178cd5f6c745809afb68bc8f7dc38b)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:34.191264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:34.258103 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21178cd5f6c745809afb68bc8f7dc38b', 'request_id': 'req-aec5effb-c4e2-47b3-8099-133c832dc843'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:34.267794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21178cd5f6c745809afb68bc8f7dc38b', 'request_id': 'req-aec5effb-c4e2-47b3-8099-133c832dc843'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:34.277617 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21178cd5f6c745809afb68bc8f7dc38b', 'request_id': 'req-aec5effb-c4e2-47b3-8099-133c832dc843'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:34.291877 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] The token cache is being invalidate because project 21178cd5f6c745809afb68bc8f7dc38b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:26:34.292172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:34.292859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:34.294417 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1345/2697] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:26:34 2022] DELETE /identity/v3/projects/21178cd5f6c745809afb68bc8f7dc38b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:34.298961 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d74a2-5577-494e-8b5e-06dd93977152 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:34.310481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d74a2-5577-494e-8b5e-06dd93977152 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:34.393003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:34.400950 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62768a7-c5ab-4128-b372-4a1098f50954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:34.402623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62768a7-c5ab-4128-b372-4a1098f50954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:34.402623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62768a7-c5ab-4128-b372-4a1098f50954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:34.403618 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62768a7-c5ab-4128-b372-4a1098f50954 admin admin] PATH_INFO: `/v3/users/841e5fd12bac450fb7bbcee30fe7e937` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:34.404763 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62768a7-c5ab-4128-b372-4a1098f50954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=841e5fd12bac450fb7bbcee30fe7e937)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:34.407085 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62768a7-c5ab-4128-b372-4a1098f50954 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:34.462478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c62768a7-c5ab-4128-b372-4a1098f50954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '841e5fd12bac450fb7bbcee30fe7e937'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:34.476098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c62768a7-c5ab-4128-b372-4a1098f50954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '841e5fd12bac450fb7bbcee30fe7e937'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:34.484988 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c62768a7-c5ab-4128-b372-4a1098f50954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '841e5fd12bac450fb7bbcee30fe7e937'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:34.485739 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c62768a7-c5ab-4128-b372-4a1098f50954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '841e5fd12bac450fb7bbcee30fe7e937'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:34.486344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c62768a7-c5ab-4128-b372-4a1098f50954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '841e5fd12bac450fb7bbcee30fe7e937'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:34.510647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1353/2698] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:26:34 2022] DELETE /identity/v3/users/841e5fd12bac450fb7bbcee30fe7e937 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:26:34.832598 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:34.834216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:34.855392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5effb-c4e2-47b3-8099-133c832dc843 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:34.895993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:34.941494 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f08210-ef39-4645-b99e-b06d069019e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:34.942216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f08210-ef39-4645-b99e-b06d069019e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:34.942564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f08210-ef39-4645-b99e-b06d069019e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:34.942928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f08210-ef39-4645-b99e-b06d069019e4 admin admin] PATH_INFO: `/v3/projects/c7902211466f4e17b0b549fed4cee5d0` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:34.944246 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f08210-ef39-4645-b99e-b06d069019e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7902211466f4e17b0b549fed4cee5d0)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:34.946340 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f08210-ef39-4645-b99e-b06d069019e4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:34.998864 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b7f08210-ef39-4645-b99e-b06d069019e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7902211466f4e17b0b549fed4cee5d0', 'request_id': 'req-b7f08210-ef39-4645-b99e-b06d069019e4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:35.006900 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b7f08210-ef39-4645-b99e-b06d069019e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7902211466f4e17b0b549fed4cee5d0', 'request_id': 'req-b7f08210-ef39-4645-b99e-b06d069019e4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:35.013327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b7f08210-ef39-4645-b99e-b06d069019e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7902211466f4e17b0b549fed4cee5d0', 'request_id': 'req-b7f08210-ef39-4645-b99e-b06d069019e4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:35.024818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b7f08210-ef39-4645-b99e-b06d069019e4 admin admin] The token cache is being invalidate because project c7902211466f4e17b0b549fed4cee5d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:26:35.025036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b7f08210-ef39-4645-b99e-b06d069019e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:35.026203 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-b7f08210-ef39-4645-b99e-b06d069019e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:35.027869 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1346/2699] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:26:34 2022] DELETE /identity/v3/projects/c7902211466f4e17b0b549fed4cee5d0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:26:35.032873 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e3f52-e775-453a-a4b8-060bddb4db5a None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:35.033162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e3f52-e775-453a-a4b8-060bddb4db5a None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:35.033437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e3f52-e775-453a-a4b8-060bddb4db5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:35.060143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-7a3e3f52-e775-453a-a4b8-060bddb4db5a None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:35.061370 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7a3e3f52-e775-453a-a4b8-060bddb4db5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:35.129686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1354/2700] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:35.138838 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908a08e-cebb-4905-889a-1c69538a2d0b None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:35.139147 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908a08e-cebb-4905-889a-1c69538a2d0b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:35.139423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908a08e-cebb-4905-889a-1c69538a2d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:35.159668 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-3908a08e-cebb-4905-889a-1c69538a2d0b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:35.160887 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3908a08e-cebb-4905-889a-1c69538a2d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:35.169718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1347/2701] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:35.174077 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e3f52-e775-453a-a4b8-060bddb4db5a None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:35.183922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e3f52-e775-453a-a4b8-060bddb4db5a None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:35.188589 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc4914-2ba7-4f77-aefd-dfa4eca3eb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:35.189207 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc4914-2ba7-4f77-aefd-dfa4eca3eb45 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:35.189490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc4914-2ba7-4f77-aefd-dfa4eca3eb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:35.189768 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc4914-2ba7-4f77-aefd-dfa4eca3eb45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:35.190423 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc4914-2ba7-4f77-aefd-dfa4eca3eb45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:26:35.190728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc4914-2ba7-4f77-aefd-dfa4eca3eb45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:35.192397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc4914-2ba7-4f77-aefd-dfa4eca3eb45 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:35.204630 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1355/2702] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:26:35.209071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908a08e-cebb-4905-889a-1c69538a2d0b None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:35.218998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908a08e-cebb-4905-889a-1c69538a2d0b None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:35.223885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2fef2-284c-4b48-bce8-36a60c09ecc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:35.224490 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2fef2-284c-4b48-bce8-36a60c09ecc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:35.224765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2fef2-284c-4b48-bce8-36a60c09ecc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:35.225031 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2fef2-284c-4b48-bce8-36a60c09ecc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:35.225700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2fef2-284c-4b48-bce8-36a60c09ecc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:35.227645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2fef2-284c-4b48-bce8-36a60c09ecc4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:35.258118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1348/2703] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:26:35 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:35.260309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc4914-2ba7-4f77-aefd-dfa4eca3eb45 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:35.270408 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc4914-2ba7-4f77-aefd-dfa4eca3eb45 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:35.329162 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ae71e-6f6f-499e-823c-fce9a17ebacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:35.329784 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ae71e-6f6f-499e-823c-fce9a17ebacf admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:35.330066 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ae71e-6f6f-499e-823c-fce9a17ebacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:35.330343 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ae71e-6f6f-499e-823c-fce9a17ebacf admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:35.331018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ae71e-6f6f-499e-823c-fce9a17ebacf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:35.333482 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ae71e-6f6f-499e-823c-fce9a17ebacf admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:35.345836 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:35.361059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1356/2704] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:26:35 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:35.365240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2fef2-284c-4b48-bce8-36a60c09ecc4 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:35.375859 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2fef2-284c-4b48-bce8-36a60c09ecc4 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:35.380605 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e59f38-0faa-4adf-981e-8bc5286225bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:35.381275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e59f38-0faa-4adf-981e-8bc5286225bf admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:35.381482 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e59f38-0faa-4adf-981e-8bc5286225bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:35.381757 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e59f38-0faa-4adf-981e-8bc5286225bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:35.382466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e59f38-0faa-4adf-981e-8bc5286225bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:35.384084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e59f38-0faa-4adf-981e-8bc5286225bf admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:35.395454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1349/2705] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:26:35.400396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ae71e-6f6f-499e-823c-fce9a17ebacf admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:35.413807 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ae71e-6f6f-499e-823c-fce9a17ebacf admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:35.418895 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da719b32-a154-42fb-8ba2-2205182d6b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:35.419544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da719b32-a154-42fb-8ba2-2205182d6b86 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:35.419827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da719b32-a154-42fb-8ba2-2205182d6b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:35.420100 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da719b32-a154-42fb-8ba2-2205182d6b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:35.421649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da719b32-a154-42fb-8ba2-2205182d6b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:35.422340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da719b32-a154-42fb-8ba2-2205182d6b86 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:35.434040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1357/2706] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:35.438423 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e59f38-0faa-4adf-981e-8bc5286225bf admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:35.449759 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e59f38-0faa-4adf-981e-8bc5286225bf admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:35.454399 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00fc92-8c4c-4b23-8fdb-3ea3994d05f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:35.455042 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00fc92-8c4c-4b23-8fdb-3ea3994d05f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:35.455370 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00fc92-8c4c-4b23-8fdb-3ea3994d05f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:35.455639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00fc92-8c4c-4b23-8fdb-3ea3994d05f9 admin admin] PATH_INFO: `/v3/projects/90524eafca00433fb75dd5b94b5cffcb/users/939d8ce4feb8444f9ff78cf2cf4cc1fb/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:35.471587 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00fc92-8c4c-4b23-8fdb-3ea3994d05f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90524eafca00433fb75dd5b94b5cffcb, user_id=939d8ce4feb8444f9ff78cf2cf4cc1fb, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:35.474260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00fc92-8c4c-4b23-8fdb-3ea3994d05f9 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:35.490008 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1350/2707] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:35 2022] PUT /identity/v3/projects/90524eafca00433fb75dd5b94b5cffcb/users/939d8ce4feb8444f9ff78cf2cf4cc1fb/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:35.503231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4136dc7-e415-44ae-bd0e-19c7b3db3bf0 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:35.503231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4136dc7-e415-44ae-bd0e-19c7b3db3bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:35.503231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4136dc7-e415-44ae-bd0e-19c7b3db3bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:35.531380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f4136dc7-e415-44ae-bd0e-19c7b3db3bf0 None None] MFA Rules not processed for user `939d8ce4feb8444f9ff78cf2cf4cc1fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:35.532625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f4136dc7-e415-44ae-bd0e-19c7b3db3bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:35.603707 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1358/2708] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:26:35.610360 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00fc92-8c4c-4b23-8fdb-3ea3994d05f9 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:35.676156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:35.692091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00fc92-8c4c-4b23-8fdb-3ea3994d05f9 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:35.698848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac793c-37cd-4001-8c3d-0aaca74e238f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:35.699570 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac793c-37cd-4001-8c3d-0aaca74e238f admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:35.699842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac793c-37cd-4001-8c3d-0aaca74e238f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:35.700111 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac793c-37cd-4001-8c3d-0aaca74e238f admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:35.700765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac793c-37cd-4001-8c3d-0aaca74e238f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:35.702698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac793c-37cd-4001-8c3d-0aaca74e238f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:35.734313 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1351/2709] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:26:35 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:35.738801 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4136dc7-e415-44ae-bd0e-19c7b3db3bf0 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:35.750810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4136dc7-e415-44ae-bd0e-19c7b3db3bf0 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:35.820445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48c55b-7853-4580-8017-8e55b6812d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:35.821059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48c55b-7853-4580-8017-8e55b6812d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:35.821337 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48c55b-7853-4580-8017-8e55b6812d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:35.821661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48c55b-7853-4580-8017-8e55b6812d8b admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:35.822303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48c55b-7853-4580-8017-8e55b6812d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:35.824767 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48c55b-7853-4580-8017-8e55b6812d8b admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:35.851078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1359/2710] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:26:35 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:35.856257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac793c-37cd-4001-8c3d-0aaca74e238f admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:35.867523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac793c-37cd-4001-8c3d-0aaca74e238f admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:35.872953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd8e81-6f50-49c6-911b-bbf4b78b9b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:35.873618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd8e81-6f50-49c6-911b-bbf4b78b9b1c admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:35.873938 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd8e81-6f50-49c6-911b-bbf4b78b9b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:35.874253 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd8e81-6f50-49c6-911b-bbf4b78b9b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:35.875029 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd8e81-6f50-49c6-911b-bbf4b78b9b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:35.877329 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd8e81-6f50-49c6-911b-bbf4b78b9b1c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:35.892495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1352/2711] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:35.897838 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48c55b-7853-4580-8017-8e55b6812d8b admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:35.915753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48c55b-7853-4580-8017-8e55b6812d8b admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:35.923047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc19de-5991-4920-8732-67d6121a9303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:35.923769 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc19de-5991-4920-8732-67d6121a9303 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:35.924050 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc19de-5991-4920-8732-67d6121a9303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:35.924320 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc19de-5991-4920-8732-67d6121a9303 admin admin] PATH_INFO: `/v3/projects/2a7c2ac5b3ee414dac9183d1786a98cd/users/71737ac89e6848ed9c6d8822fd444a8d/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:35.942576 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc19de-5991-4920-8732-67d6121a9303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7c2ac5b3ee414dac9183d1786a98cd, user_id=71737ac89e6848ed9c6d8822fd444a8d, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:35.945274 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc19de-5991-4920-8732-67d6121a9303 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:35.962132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1360/2712] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:35 2022] PUT /identity/v3/projects/2a7c2ac5b3ee414dac9183d1786a98cd/users/71737ac89e6848ed9c6d8822fd444a8d/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:35.967279 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd8e81-6f50-49c6-911b-bbf4b78b9b1c admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:35.980107 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd8e81-6f50-49c6-911b-bbf4b78b9b1c admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:36.043337 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1dc8e6-73b6-47c9-9f94-4ab46ef36991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:36.043993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dc8e6-73b6-47c9-9f94-4ab46ef36991 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:36.044268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dc8e6-73b6-47c9-9f94-4ab46ef36991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:36.044542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dc8e6-73b6-47c9-9f94-4ab46ef36991 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:36.045195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1dc8e6-73b6-47c9-9f94-4ab46ef36991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:36.046837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1dc8e6-73b6-47c9-9f94-4ab46ef36991 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:36.057903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1353/2713] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:36.062456 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc19de-5991-4920-8732-67d6121a9303 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:36.074401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc19de-5991-4920-8732-67d6121a9303 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:36.080626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd78743-2acd-4054-a303-8844d18cd38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:36.081305 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd78743-2acd-4054-a303-8844d18cd38b admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:36.081585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd78743-2acd-4054-a303-8844d18cd38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:36.081862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd78743-2acd-4054-a303-8844d18cd38b admin admin] PATH_INFO: `/v3/projects/2a7c2ac5b3ee414dac9183d1786a98cd/users/71737ac89e6848ed9c6d8822fd444a8d/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:36.083834 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd78743-2acd-4054-a303-8844d18cd38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7c2ac5b3ee414dac9183d1786a98cd, user_id=71737ac89e6848ed9c6d8822fd444a8d, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:36.086518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd78743-2acd-4054-a303-8844d18cd38b admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:36.101631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1361/2714] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:36 2022] PUT /identity/v3/projects/2a7c2ac5b3ee414dac9183d1786a98cd/users/71737ac89e6848ed9c6d8822fd444a8d/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:36.113012 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a48cf9-4e45-468c-aa61-c6a8a9ea8363 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:36.113278 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a48cf9-4e45-468c-aa61-c6a8a9ea8363 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:36.113562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a48cf9-4e45-468c-aa61-c6a8a9ea8363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:36.146470 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-d4a48cf9-4e45-468c-aa61-c6a8a9ea8363 None None] MFA Rules not processed for user `71737ac89e6848ed9c6d8822fd444a8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:36.147751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d4a48cf9-4e45-468c-aa61-c6a8a9ea8363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:36.231144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1354/2715] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:26:36.243599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd78743-2acd-4054-a303-8844d18cd38b admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:36.245378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-dbd78743-2acd-4054-a303-8844d18cd38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:36.274625 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:36.306213 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd78743-2acd-4054-a303-8844d18cd38b admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:36.309948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea3379-0f30-45ad-8918-762fdc441209 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:36.310564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea3379-0f30-45ad-8918-762fdc441209 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:36.310843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea3379-0f30-45ad-8918-762fdc441209 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:36.311139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea3379-0f30-45ad-8918-762fdc441209 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:36.320749 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea3379-0f30-45ad-8918-762fdc441209 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:36.323036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea3379-0f30-45ad-8918-762fdc441209 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:36.333114 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1362/2716] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:26:36.362384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a48cf9-4e45-468c-aa61-c6a8a9ea8363 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:36.413534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:36.429050 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a48cf9-4e45-468c-aa61-c6a8a9ea8363 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:36.434368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:36.435017 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:36.435340 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:36.435622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] PATH_INFO: `/v3/users/939d8ce4feb8444f9ff78cf2cf4cc1fb` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:36.436767 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] RBAC: Authorizing `identity:delete_user(user_id=939d8ce4feb8444f9ff78cf2cf4cc1fb)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:36.438717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:36.490099 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939d8ce4feb8444f9ff78cf2cf4cc1fb'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:36.500057 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939d8ce4feb8444f9ff78cf2cf4cc1fb'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:36.500823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939d8ce4feb8444f9ff78cf2cf4cc1fb'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:36.501598 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939d8ce4feb8444f9ff78cf2cf4cc1fb'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:36.514717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939d8ce4feb8444f9ff78cf2cf4cc1fb'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:36.527380 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1355/2717] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:26:36 2022] DELETE /identity/v3/users/939d8ce4feb8444f9ff78cf2cf4cc1fb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:36.532072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826ffe1-85ac-4bf1-82da-fc803ea3844b None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:36.532623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826ffe1-85ac-4bf1-82da-fc803ea3844b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:36.532623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826ffe1-85ac-4bf1-82da-fc803ea3844b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:36.552363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-4826ffe1-85ac-4bf1-82da-fc803ea3844b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:36.553571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4826ffe1-85ac-4bf1-82da-fc803ea3844b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:36.618028 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1363/2718] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:36.628638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:36.630065 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:36.696835 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a64a2-41d5-43a3-a7f1-24d6c6bc385b admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:36.701305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b56e1-d5b7-4ddf-a197-e5de29ee384f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:36.701936 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b56e1-d5b7-4ddf-a197-e5de29ee384f service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:36.704052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b56e1-d5b7-4ddf-a197-e5de29ee384f service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:36.704052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b56e1-d5b7-4ddf-a197-e5de29ee384f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:36.721393 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b56e1-d5b7-4ddf-a197-e5de29ee384f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:36.725406 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b56e1-d5b7-4ddf-a197-e5de29ee384f service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:36.746913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1356/2719] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:37.057179 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826ffe1-85ac-4bf1-82da-fc803ea3844b None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:37.058606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4826ffe1-85ac-4bf1-82da-fc803ea3844b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:37.078181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826ffe1-85ac-4bf1-82da-fc803ea3844b None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:37.087143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:37.087143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:37.087143 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:37.087552 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] PATH_INFO: `/v3/projects/90524eafca00433fb75dd5b94b5cffcb` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:37.089775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90524eafca00433fb75dd5b94b5cffcb)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:37.093680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:37.147089 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90524eafca00433fb75dd5b94b5cffcb', 'request_id': 'req-6b51d3dd-e2e2-4230-9aea-627b9731b829'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.154639 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90524eafca00433fb75dd5b94b5cffcb', 'request_id': 'req-6b51d3dd-e2e2-4230-9aea-627b9731b829'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.160571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90524eafca00433fb75dd5b94b5cffcb', 'request_id': 'req-6b51d3dd-e2e2-4230-9aea-627b9731b829'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.174714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] The token cache is being invalidate because project 90524eafca00433fb75dd5b94b5cffcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:26:37.175223 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.176016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.177745 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1364/2720] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:26:37 2022] DELETE /identity/v3/projects/90524eafca00433fb75dd5b94b5cffcb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:37.182615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b56e1-d5b7-4ddf-a197-e5de29ee384f service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:37.193319 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b56e1-d5b7-4ddf-a197-e5de29ee384f service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:37.256748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91914d2f-8803-4e39-ae67-1bc77aadf0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:37.257443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91914d2f-8803-4e39-ae67-1bc77aadf0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:37.257728 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91914d2f-8803-4e39-ae67-1bc77aadf0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:37.258009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91914d2f-8803-4e39-ae67-1bc77aadf0d6 admin admin] PATH_INFO: `/v3/users/71737ac89e6848ed9c6d8822fd444a8d` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:37.259262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91914d2f-8803-4e39-ae67-1bc77aadf0d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71737ac89e6848ed9c6d8822fd444a8d)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:37.261820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91914d2f-8803-4e39-ae67-1bc77aadf0d6 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:37.278751 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:37.339531 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-91914d2f-8803-4e39-ae67-1bc77aadf0d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71737ac89e6848ed9c6d8822fd444a8d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.351039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-91914d2f-8803-4e39-ae67-1bc77aadf0d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71737ac89e6848ed9c6d8822fd444a8d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.351818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-91914d2f-8803-4e39-ae67-1bc77aadf0d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71737ac89e6848ed9c6d8822fd444a8d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.352443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-91914d2f-8803-4e39-ae67-1bc77aadf0d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71737ac89e6848ed9c6d8822fd444a8d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.366250 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-91914d2f-8803-4e39-ae67-1bc77aadf0d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71737ac89e6848ed9c6d8822fd444a8d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.384121 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1357/2721] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:26:37 2022] DELETE /identity/v3/users/71737ac89e6848ed9c6d8822fd444a8d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:26:37.675706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:37.677232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:37.696467 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51d3dd-e2e2-4230-9aea-627b9731b829 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:37.750119 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:37.762099 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:37.762740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:37.763017 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:37.763317 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44 admin admin] PATH_INFO: `/v3/projects/2a7c2ac5b3ee414dac9183d1786a98cd` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:37.764465 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a7c2ac5b3ee414dac9183d1786a98cd)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:37.766377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:37.820323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a7c2ac5b3ee414dac9183d1786a98cd', 'request_id': 'req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.832265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a7c2ac5b3ee414dac9183d1786a98cd', 'request_id': 'req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.842311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a7c2ac5b3ee414dac9183d1786a98cd', 'request_id': 'req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.862881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44 admin admin] The token cache is being invalidate because project 2a7c2ac5b3ee414dac9183d1786a98cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:26:37.863454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.864513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-65ec7a26-25e8-4e24-8f4c-6b5ba9056e44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:37.867130 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1365/2722] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:26:37 2022] DELETE /identity/v3/projects/2a7c2ac5b3ee414dac9183d1786a98cd => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:37.872886 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b1f0c-cb63-43d5-a3f1-d9863e3a6f14 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:37.873171 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b1f0c-cb63-43d5-a3f1-d9863e3a6f14 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:37.873448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b1f0c-cb63-43d5-a3f1-d9863e3a6f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:37.899091 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-9c7b1f0c-cb63-43d5-a3f1-d9863e3a6f14 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:37.900983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-9c7b1f0c-cb63-43d5-a3f1-d9863e3a6f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:37.973675 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1358/2723] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:37.986338 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c942d-1004-44ce-bc4e-d4f2c7063d08 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:37.986844 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c942d-1004-44ce-bc4e-d4f2c7063d08 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:37.987394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c942d-1004-44ce-bc4e-d4f2c7063d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:38.011833 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a26c942d-1004-44ce-bc4e-d4f2c7063d08 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:38.013129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a26c942d-1004-44ce-bc4e-d4f2c7063d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:38.021218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1366/2724] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:38.026322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b1f0c-cb63-43d5-a3f1-d9863e3a6f14 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:38.036747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b1f0c-cb63-43d5-a3f1-d9863e3a6f14 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:38.043039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04477dbc-7942-4a69-bf54-4ac91f719707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:38.043749 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04477dbc-7942-4a69-bf54-4ac91f719707 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:38.044073 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04477dbc-7942-4a69-bf54-4ac91f719707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:38.044341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04477dbc-7942-4a69-bf54-4ac91f719707 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:38.045023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04477dbc-7942-4a69-bf54-4ac91f719707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:26:38.045320 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04477dbc-7942-4a69-bf54-4ac91f719707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:38.047175 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04477dbc-7942-4a69-bf54-4ac91f719707 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:38.060354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1359/2725] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:26:38.064774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c942d-1004-44ce-bc4e-d4f2c7063d08 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:38.074688 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c942d-1004-44ce-bc4e-d4f2c7063d08 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:38.079476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf1fe9-6d65-424f-8444-daae272ffe6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:38.080082 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf1fe9-6d65-424f-8444-daae272ffe6c admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:38.080864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf1fe9-6d65-424f-8444-daae272ffe6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:38.081136 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf1fe9-6d65-424f-8444-daae272ffe6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:38.081795 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf1fe9-6d65-424f-8444-daae272ffe6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:38.084008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf1fe9-6d65-424f-8444-daae272ffe6c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:38.119864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1367/2726] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:26:38 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:38.124585 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04477dbc-7942-4a69-bf54-4ac91f719707 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:38.137502 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04477dbc-7942-4a69-bf54-4ac91f719707 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:38.198691 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6119f1c-ef6b-4818-a635-bdafff6405bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:38.199415 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6119f1c-ef6b-4818-a635-bdafff6405bc admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:38.199618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6119f1c-ef6b-4818-a635-bdafff6405bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:38.199894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6119f1c-ef6b-4818-a635-bdafff6405bc admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:38.200538 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6119f1c-ef6b-4818-a635-bdafff6405bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:38.202493 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6119f1c-ef6b-4818-a635-bdafff6405bc admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:38.226638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1360/2727] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:26:38 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:38.231843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf1fe9-6d65-424f-8444-daae272ffe6c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:38.242559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf1fe9-6d65-424f-8444-daae272ffe6c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:38.247397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d650e4-52e0-4d23-8b81-ec72edd15416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:38.248018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d650e4-52e0-4d23-8b81-ec72edd15416 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:38.248296 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d650e4-52e0-4d23-8b81-ec72edd15416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:38.248579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d650e4-52e0-4d23-8b81-ec72edd15416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:38.249225 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d650e4-52e0-4d23-8b81-ec72edd15416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:38.250817 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d650e4-52e0-4d23-8b81-ec72edd15416 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:38.261449 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1368/2728] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:38.265437 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6119f1c-ef6b-4818-a635-bdafff6405bc admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:38.277090 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6119f1c-ef6b-4818-a635-bdafff6405bc admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:38.283874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89542d4a-aa80-4591-a151-3f94ad844067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:38.284506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89542d4a-aa80-4591-a151-3f94ad844067 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:38.284808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89542d4a-aa80-4591-a151-3f94ad844067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:38.285080 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89542d4a-aa80-4591-a151-3f94ad844067 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:38.285760 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89542d4a-aa80-4591-a151-3f94ad844067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:38.287426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89542d4a-aa80-4591-a151-3f94ad844067 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:38.288965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:38.302687 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1361/2729] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:38.306604 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d650e4-52e0-4d23-8b81-ec72edd15416 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:38.320571 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d650e4-52e0-4d23-8b81-ec72edd15416 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:38.326226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2412a4-30bf-41a0-9d72-bdd62f29d07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:38.326826 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2412a4-30bf-41a0-9d72-bdd62f29d07b admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:38.327129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2412a4-30bf-41a0-9d72-bdd62f29d07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:38.327426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2412a4-30bf-41a0-9d72-bdd62f29d07b admin admin] PATH_INFO: `/v3/projects/8a6a8b38677d4df2b2f5b647008acccb/users/40aeae0e689843ed916583f5a3f12446/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:38.346732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2412a4-30bf-41a0-9d72-bdd62f29d07b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6a8b38677d4df2b2f5b647008acccb, user_id=40aeae0e689843ed916583f5a3f12446, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:38.350054 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2412a4-30bf-41a0-9d72-bdd62f29d07b admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:38.369949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1369/2730] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:38 2022] PUT /identity/v3/projects/8a6a8b38677d4df2b2f5b647008acccb/users/40aeae0e689843ed916583f5a3f12446/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:26:38.381024 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7605c-08fa-48fc-bc87-3c54be84292c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:38.381368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7605c-08fa-48fc-bc87-3c54be84292c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:38.381722 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7605c-08fa-48fc-bc87-3c54be84292c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:38.415355 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ecc7605c-08fa-48fc-bc87-3c54be84292c None None] MFA Rules not processed for user `40aeae0e689843ed916583f5a3f12446`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:38.416424 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ecc7605c-08fa-48fc-bc87-3c54be84292c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:38.502193 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1362/2731] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:26:38.507452 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2412a4-30bf-41a0-9d72-bdd62f29d07b admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:38.591038 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2412a4-30bf-41a0-9d72-bdd62f29d07b admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:38.596535 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420f9ac-8d2b-40e6-97ee-8b258dbe9c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:38.597153 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420f9ac-8d2b-40e6-97ee-8b258dbe9c65 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:38.597439 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420f9ac-8d2b-40e6-97ee-8b258dbe9c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:38.597714 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420f9ac-8d2b-40e6-97ee-8b258dbe9c65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:38.598394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420f9ac-8d2b-40e6-97ee-8b258dbe9c65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:38.600848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420f9ac-8d2b-40e6-97ee-8b258dbe9c65 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:38.637803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1370/2732] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:26:38 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:26:38.639597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7605c-08fa-48fc-bc87-3c54be84292c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:38.652222 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7605c-08fa-48fc-bc87-3c54be84292c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:38.842237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0c9b5-55ae-4173-977b-e794d853c174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:38.842773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0c9b5-55ae-4173-977b-e794d853c174 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:38.843629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0c9b5-55ae-4173-977b-e794d853c174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:38.843629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0c9b5-55ae-4173-977b-e794d853c174 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:38.844008 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0c9b5-55ae-4173-977b-e794d853c174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:38.845972 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0c9b5-55ae-4173-977b-e794d853c174 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:38.870396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1363/2733] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:26:38 2022] POST /identity/v3/users => generated 300 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:38.875481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420f9ac-8d2b-40e6-97ee-8b258dbe9c65 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:38.882266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:38.888649 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420f9ac-8d2b-40e6-97ee-8b258dbe9c65 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:38.897369 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1af4c-d4e7-464b-a8f8-6e1f355468cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:38.898378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1af4c-d4e7-464b-a8f8-6e1f355468cf admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:38.898775 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1af4c-d4e7-464b-a8f8-6e1f355468cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:38.901579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1af4c-d4e7-464b-a8f8-6e1f355468cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:38.901579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1af4c-d4e7-464b-a8f8-6e1f355468cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:38.904106 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1af4c-d4e7-464b-a8f8-6e1f355468cf admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:38.917780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1371/2734] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:38.923188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0c9b5-55ae-4173-977b-e794d853c174 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:38.934221 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0c9b5-55ae-4173-977b-e794d853c174 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:38.948234 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a96c5-479c-4dba-bbd2-ddd5a06cf91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:38.948903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a96c5-479c-4dba-bbd2-ddd5a06cf91d admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:38.949180 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a96c5-479c-4dba-bbd2-ddd5a06cf91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:38.949458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a96c5-479c-4dba-bbd2-ddd5a06cf91d admin admin] PATH_INFO: `/v3/projects/c3e3967f84d44461bae548fe3f6268e7/users/56336dd693384462a6cb8dccf65aee2c/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:38.966453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a96c5-479c-4dba-bbd2-ddd5a06cf91d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e3967f84d44461bae548fe3f6268e7, user_id=56336dd693384462a6cb8dccf65aee2c, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:38.969842 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a96c5-479c-4dba-bbd2-ddd5a06cf91d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:38.987364 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1364/2735] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:38 2022] PUT /identity/v3/projects/c3e3967f84d44461bae548fe3f6268e7/users/56336dd693384462a6cb8dccf65aee2c/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:38.991520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1af4c-d4e7-464b-a8f8-6e1f355468cf admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:39.002577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1af4c-d4e7-464b-a8f8-6e1f355468cf admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:39.078734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e1256-7c92-4f60-84dd-fac1bbdf8b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:39.079396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e1256-7c92-4f60-84dd-fac1bbdf8b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:39.079678 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e1256-7c92-4f60-84dd-fac1bbdf8b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:39.079954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e1256-7c92-4f60-84dd-fac1bbdf8b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:39.080615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e1256-7c92-4f60-84dd-fac1bbdf8b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:39.082218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e1256-7c92-4f60-84dd-fac1bbdf8b89 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:39.104897 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1372/2736] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:39.110388 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a96c5-479c-4dba-bbd2-ddd5a06cf91d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:39.124396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a96c5-479c-4dba-bbd2-ddd5a06cf91d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:39.130229 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9b95f-9e77-44c0-ae9c-5f29c31a2c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:39.130907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9b95f-9e77-44c0-ae9c-5f29c31a2c64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:39.131218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9b95f-9e77-44c0-ae9c-5f29c31a2c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:39.131494 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9b95f-9e77-44c0-ae9c-5f29c31a2c64 admin admin] PATH_INFO: `/v3/projects/c3e3967f84d44461bae548fe3f6268e7/users/56336dd693384462a6cb8dccf65aee2c/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:39.135569 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9b95f-9e77-44c0-ae9c-5f29c31a2c64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e3967f84d44461bae548fe3f6268e7, user_id=56336dd693384462a6cb8dccf65aee2c, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:39.138201 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9b95f-9e77-44c0-ae9c-5f29c31a2c64 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:39.151603 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1365/2737] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:39 2022] PUT /identity/v3/projects/c3e3967f84d44461bae548fe3f6268e7/users/56336dd693384462a6cb8dccf65aee2c/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:39.164151 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee9cb0-c476-4889-8d64-65d9931f0971 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:39.164590 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee9cb0-c476-4889-8d64-65d9931f0971 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:39.165007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee9cb0-c476-4889-8d64-65d9931f0971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:39.200174 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-6cee9cb0-c476-4889-8d64-65d9931f0971 None None] MFA Rules not processed for user `56336dd693384462a6cb8dccf65aee2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:39.201425 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-6cee9cb0-c476-4889-8d64-65d9931f0971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:39.286074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1373/2738] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:39.298734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9b95f-9e77-44c0-ae9c-5f29c31a2c64 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:39.300395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-fbe9b95f-9e77-44c0-ae9c-5f29c31a2c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:39.365064 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:39.385754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9b95f-9e77-44c0-ae9c-5f29c31a2c64 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:39.391959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a130b-0a0c-4ca0-8299-23ac11dddc78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:39.392967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a130b-0a0c-4ca0-8299-23ac11dddc78 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:39.393440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a130b-0a0c-4ca0-8299-23ac11dddc78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:39.393899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a130b-0a0c-4ca0-8299-23ac11dddc78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:39.410219 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a130b-0a0c-4ca0-8299-23ac11dddc78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:39.412915 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a130b-0a0c-4ca0-8299-23ac11dddc78 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:39.430022 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1366/2739] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:39.502597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee9cb0-c476-4889-8d64-65d9931f0971 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:39.570131 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee9cb0-c476-4889-8d64-65d9931f0971 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:39.575404 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:39.576464 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:39.576748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:39.577026 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] PATH_INFO: `/v3/users/40aeae0e689843ed916583f5a3f12446` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:39.578181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=40aeae0e689843ed916583f5a3f12446)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:39.580171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:39.595632 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:39.637938 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40aeae0e689843ed916583f5a3f12446'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:39.650063 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40aeae0e689843ed916583f5a3f12446'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:39.660388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40aeae0e689843ed916583f5a3f12446'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:39.661161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40aeae0e689843ed916583f5a3f12446'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:39.661748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40aeae0e689843ed916583f5a3f12446'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:39.689150 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1374/2740] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:26:39 2022] DELETE /identity/v3/users/40aeae0e689843ed916583f5a3f12446 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:39.694290 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28569496-2a52-4093-b78f-3032202116cf None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:39.694575 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28569496-2a52-4093-b78f-3032202116cf None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:39.694846 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28569496-2a52-4093-b78f-3032202116cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:39.716894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-28569496-2a52-4093-b78f-3032202116cf None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:39.718140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-28569496-2a52-4093-b78f-3032202116cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:39.805411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1367/2741] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:39.816551 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:39.819522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:39.923355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae469966-e655-459e-8e31-afab137ed7cb admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:39.927680 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f8c47-b64e-47e9-8051-07a7da505e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:39.928312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f8c47-b64e-47e9-8051-07a7da505e53 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:39.928593 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f8c47-b64e-47e9-8051-07a7da505e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:39.928870 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f8c47-b64e-47e9-8051-07a7da505e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:39.939431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f8c47-b64e-47e9-8051-07a7da505e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:39.940899 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f8c47-b64e-47e9-8051-07a7da505e53 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:39.953434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1375/2742] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:40.260390 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28569496-2a52-4093-b78f-3032202116cf None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:40.261908 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-28569496-2a52-4093-b78f-3032202116cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:40.274269 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28569496-2a52-4093-b78f-3032202116cf None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:40.279238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:40.279843 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:40.280135 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:40.280378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] PATH_INFO: `/v3/projects/8a6a8b38677d4df2b2f5b647008acccb` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:40.281536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a6a8b38677d4df2b2f5b647008acccb)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:40.284644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:40.338256 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a6a8b38677d4df2b2f5b647008acccb', 'request_id': 'req-76581285-3766-40ff-b48a-e8b161d528e2'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:40.346170 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a6a8b38677d4df2b2f5b647008acccb', 'request_id': 'req-76581285-3766-40ff-b48a-e8b161d528e2'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:40.352109 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a6a8b38677d4df2b2f5b647008acccb', 'request_id': 'req-76581285-3766-40ff-b48a-e8b161d528e2'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:40.363794 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] The token cache is being invalidate because project 8a6a8b38677d4df2b2f5b647008acccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:26:40.364075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:40.364730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:40.366212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1368/2743] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:26:40 2022] DELETE /identity/v3/projects/8a6a8b38677d4df2b2f5b647008acccb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:40.371345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f8c47-b64e-47e9-8051-07a7da505e53 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:40.381167 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f8c47-b64e-47e9-8051-07a7da505e53 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:40.439195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad913d18-dac2-4a68-a6b8-df7a0513998c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:40.439688 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad913d18-dac2-4a68-a6b8-df7a0513998c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:40.439965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad913d18-dac2-4a68-a6b8-df7a0513998c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:40.440237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad913d18-dac2-4a68-a6b8-df7a0513998c admin admin] PATH_INFO: `/v3/users/56336dd693384462a6cb8dccf65aee2c` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:40.441478 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad913d18-dac2-4a68-a6b8-df7a0513998c admin admin] RBAC: Authorizing `identity:delete_user(user_id=56336dd693384462a6cb8dccf65aee2c)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:40.443443 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad913d18-dac2-4a68-a6b8-df7a0513998c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:40.485037 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:40.496426 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ad913d18-dac2-4a68-a6b8-df7a0513998c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56336dd693384462a6cb8dccf65aee2c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:40.509245 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ad913d18-dac2-4a68-a6b8-df7a0513998c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56336dd693384462a6cb8dccf65aee2c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:40.517983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ad913d18-dac2-4a68-a6b8-df7a0513998c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56336dd693384462a6cb8dccf65aee2c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:40.518724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ad913d18-dac2-4a68-a6b8-df7a0513998c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56336dd693384462a6cb8dccf65aee2c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:40.519380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-ad913d18-dac2-4a68-a6b8-df7a0513998c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56336dd693384462a6cb8dccf65aee2c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:40.537118 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1376/2744] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:26:40 2022] DELETE /identity/v3/users/56336dd693384462a6cb8dccf65aee2c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:40.840599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:40.842230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:40.858065 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76581285-3766-40ff-b48a-e8b161d528e2 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:40.952434 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd5b17-667c-4490-ab4b-37a81cabf13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:40.952998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd5b17-667c-4490-ab4b-37a81cabf13c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:40.953277 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd5b17-667c-4490-ab4b-37a81cabf13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:40.953559 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd5b17-667c-4490-ab4b-37a81cabf13c admin admin] PATH_INFO: `/v3/projects/c3e3967f84d44461bae548fe3f6268e7` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:40.954795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd5b17-667c-4490-ab4b-37a81cabf13c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3e3967f84d44461bae548fe3f6268e7)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:40.956739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd5b17-667c-4490-ab4b-37a81cabf13c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:40.964288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:41.020936 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-70fd5b17-667c-4490-ab4b-37a81cabf13c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e3967f84d44461bae548fe3f6268e7', 'request_id': 'req-70fd5b17-667c-4490-ab4b-37a81cabf13c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:41.030875 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-70fd5b17-667c-4490-ab4b-37a81cabf13c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e3967f84d44461bae548fe3f6268e7', 'request_id': 'req-70fd5b17-667c-4490-ab4b-37a81cabf13c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:41.037338 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-70fd5b17-667c-4490-ab4b-37a81cabf13c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e3967f84d44461bae548fe3f6268e7', 'request_id': 'req-70fd5b17-667c-4490-ab4b-37a81cabf13c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:41.050568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-70fd5b17-667c-4490-ab4b-37a81cabf13c admin admin] The token cache is being invalidate because project c3e3967f84d44461bae548fe3f6268e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:26:41.051085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-70fd5b17-667c-4490-ab4b-37a81cabf13c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:41.051755 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-70fd5b17-667c-4490-ab4b-37a81cabf13c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:41.054040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1369/2745] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:26:40 2022] DELETE /identity/v3/projects/c3e3967f84d44461bae548fe3f6268e7 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:41.060139 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495ba8-316e-41c9-96dc-122e63ccd9ef None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:41.060955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495ba8-316e-41c9-96dc-122e63ccd9ef None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:41.060955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52495ba8-316e-41c9-96dc-122e63ccd9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:41.087084 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-52495ba8-316e-41c9-96dc-122e63ccd9ef None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:41.089799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-52495ba8-316e-41c9-96dc-122e63ccd9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:41.182192 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1377/2746] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:41.190071 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f9ab7-19b3-43cb-832f-8e9ddf309780 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:41.190472 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f9ab7-19b3-43cb-832f-8e9ddf309780 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:41.190623 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f9ab7-19b3-43cb-832f-8e9ddf309780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:41.211849 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-1a6f9ab7-19b3-43cb-832f-8e9ddf309780 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:41.213186 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1a6f9ab7-19b3-43cb-832f-8e9ddf309780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:41.222051 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1370/2747] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:26:41.226540 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495ba8-316e-41c9-96dc-122e63ccd9ef None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:41.240155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52495ba8-316e-41c9-96dc-122e63ccd9ef None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:41.246728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bd5cc-4438-48b7-9060-de076fbef537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:41.247390 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bd5cc-4438-48b7-9060-de076fbef537 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:41.247698 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bd5cc-4438-48b7-9060-de076fbef537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:41.247977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bd5cc-4438-48b7-9060-de076fbef537 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:41.248641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bd5cc-4438-48b7-9060-de076fbef537 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:26:41.248980 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bd5cc-4438-48b7-9060-de076fbef537 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:41.250597 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bd5cc-4438-48b7-9060-de076fbef537 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:41.266676 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1378/2748] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:26:41.273963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f9ab7-19b3-43cb-832f-8e9ddf309780 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:41.291028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f9ab7-19b3-43cb-832f-8e9ddf309780 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:41.297537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486ed65-9053-4757-b453-7f4b0ecec5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:41.298394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486ed65-9053-4757-b453-7f4b0ecec5ee admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:41.298826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486ed65-9053-4757-b453-7f4b0ecec5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:41.299324 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486ed65-9053-4757-b453-7f4b0ecec5ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:41.300440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486ed65-9053-4757-b453-7f4b0ecec5ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:41.303478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486ed65-9053-4757-b453-7f4b0ecec5ee admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:41.346354 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1371/2749] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:26:41 2022] POST /identity/v3/projects => generated 387 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:41.350813 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bd5cc-4438-48b7-9060-de076fbef537 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:41.362101 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bd5cc-4438-48b7-9060-de076fbef537 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:41.428508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6daca-d9bf-4b84-9824-e0ff8da89fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:41.429219 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6daca-d9bf-4b84-9824-e0ff8da89fae admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:41.429560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6daca-d9bf-4b84-9824-e0ff8da89fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:41.429847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6daca-d9bf-4b84-9824-e0ff8da89fae admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:41.430626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6daca-d9bf-4b84-9824-e0ff8da89fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:41.433504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6daca-d9bf-4b84-9824-e0ff8da89fae admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:41.461028 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1379/2750] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:26:41 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:41.465756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486ed65-9053-4757-b453-7f4b0ecec5ee admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:41.479641 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486ed65-9053-4757-b453-7f4b0ecec5ee admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:41.484905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926b7ad-68e6-43f4-9332-a52d35267a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:41.485526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926b7ad-68e6-43f4-9332-a52d35267a49 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:41.485799 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926b7ad-68e6-43f4-9332-a52d35267a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:41.486063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926b7ad-68e6-43f4-9332-a52d35267a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:41.486705 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926b7ad-68e6-43f4-9332-a52d35267a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:41.488335 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926b7ad-68e6-43f4-9332-a52d35267a49 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:41.499772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1372/2751] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:41.504379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6daca-d9bf-4b84-9824-e0ff8da89fae admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:41.516968 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6daca-d9bf-4b84-9824-e0ff8da89fae admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:41.521882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eeb40-ec42-42de-be80-5af650b93362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:41.522504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eeb40-ec42-42de-be80-5af650b93362 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:41.522825 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eeb40-ec42-42de-be80-5af650b93362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:41.523142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eeb40-ec42-42de-be80-5af650b93362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:41.523805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2eeb40-ec42-42de-be80-5af650b93362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:41.525613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2eeb40-ec42-42de-be80-5af650b93362 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:41.537554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1380/2752] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:26:41.542991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926b7ad-68e6-43f4-9332-a52d35267a49 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:41.553676 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926b7ad-68e6-43f4-9332-a52d35267a49 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:41.560636 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439d74f-32b3-4a66-83be-9608d4c4752f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:41.561422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439d74f-32b3-4a66-83be-9608d4c4752f admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:41.561815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439d74f-32b3-4a66-83be-9608d4c4752f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:41.562212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439d74f-32b3-4a66-83be-9608d4c4752f admin admin] PATH_INFO: `/v3/projects/10dea22ba13d4bdeaedce5af9280597b/users/752eeab04c114fc5a87e53f502086dd0/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:41.579384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439d74f-32b3-4a66-83be-9608d4c4752f admin admin] RBAC: Authorizing `identity:create_grant(project_id=10dea22ba13d4bdeaedce5af9280597b, user_id=752eeab04c114fc5a87e53f502086dd0, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:41.582055 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439d74f-32b3-4a66-83be-9608d4c4752f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:41.596378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1373/2753] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:41 2022] PUT /identity/v3/projects/10dea22ba13d4bdeaedce5af9280597b/users/752eeab04c114fc5a87e53f502086dd0/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:41.607623 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893f2f5-16f3-4de3-950e-f543266830c3 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:41.607906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893f2f5-16f3-4de3-950e-f543266830c3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:41.608180 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893f2f5-16f3-4de3-950e-f543266830c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:41.638594 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-5893f2f5-16f3-4de3-950e-f543266830c3 None None] MFA Rules not processed for user `752eeab04c114fc5a87e53f502086dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:41.640045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5893f2f5-16f3-4de3-950e-f543266830c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:41.650882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:41.721856 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1381/2754] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:41.726836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439d74f-32b3-4a66-83be-9608d4c4752f admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:41.793910 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439d74f-32b3-4a66-83be-9608d4c4752f admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:41.798759 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68633e4b-5b5a-48b5-85b6-7bd6e4e41fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:41.799395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68633e4b-5b5a-48b5-85b6-7bd6e4e41fdf admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:41.799678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68633e4b-5b5a-48b5-85b6-7bd6e4e41fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:41.799953 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68633e4b-5b5a-48b5-85b6-7bd6e4e41fdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:41.800588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68633e4b-5b5a-48b5-85b6-7bd6e4e41fdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:41.802503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68633e4b-5b5a-48b5-85b6-7bd6e4e41fdf admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:41.831010 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1374/2755] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:26:41 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:41.835448 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893f2f5-16f3-4de3-950e-f543266830c3 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:41.847427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893f2f5-16f3-4de3-950e-f543266830c3 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:41.918921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c4c61-c66e-480a-a855-7ba2ef5d8bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:41.919601 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c4c61-c66e-480a-a855-7ba2ef5d8bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:41.919915 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c4c61-c66e-480a-a855-7ba2ef5d8bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:41.920278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c4c61-c66e-480a-a855-7ba2ef5d8bf1 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:41.920935 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c4c61-c66e-480a-a855-7ba2ef5d8bf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:41.922853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c4c61-c66e-480a-a855-7ba2ef5d8bf1 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:41.945299 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1382/2756] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:26:41 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:26:41.949960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68633e4b-5b5a-48b5-85b6-7bd6e4e41fdf admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:41.960348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68633e4b-5b5a-48b5-85b6-7bd6e4e41fdf admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:41.965880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231336c-4432-41cf-b858-fe7ff7888176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:41.966489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231336c-4432-41cf-b858-fe7ff7888176 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:41.966765 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231336c-4432-41cf-b858-fe7ff7888176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:41.967032 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231336c-4432-41cf-b858-fe7ff7888176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:41.967708 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231336c-4432-41cf-b858-fe7ff7888176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:41.969303 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231336c-4432-41cf-b858-fe7ff7888176 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:41.981267 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1375/2757] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:41.985930 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c4c61-c66e-480a-a855-7ba2ef5d8bf1 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:41.996303 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c4c61-c66e-480a-a855-7ba2ef5d8bf1 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:42.001075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c1183-6a83-43a8-a748-a90ac19aa2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:42.001739 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c1183-6a83-43a8-a748-a90ac19aa2fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:42.002040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c1183-6a83-43a8-a748-a90ac19aa2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:42.002318 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c1183-6a83-43a8-a748-a90ac19aa2fe admin admin] PATH_INFO: `/v3/projects/c0ba136124fb49f1b64f1232fc0d3811/users/afbea107ec7747ceb4ee07fe20e7a468/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:42.019517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c1183-6a83-43a8-a748-a90ac19aa2fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ba136124fb49f1b64f1232fc0d3811, user_id=afbea107ec7747ceb4ee07fe20e7a468, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:42.022266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c1183-6a83-43a8-a748-a90ac19aa2fe admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:42.040654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1383/2758] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:41 2022] PUT /identity/v3/projects/c0ba136124fb49f1b64f1232fc0d3811/users/afbea107ec7747ceb4ee07fe20e7a468/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:42.045685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231336c-4432-41cf-b858-fe7ff7888176 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:42.052368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:42.058122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231336c-4432-41cf-b858-fe7ff7888176 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:42.127039 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ea297-74d1-4d5f-a0fe-3a9f973e466c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:42.127686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ea297-74d1-4d5f-a0fe-3a9f973e466c admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:42.127965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ea297-74d1-4d5f-a0fe-3a9f973e466c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:42.128246 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ea297-74d1-4d5f-a0fe-3a9f973e466c admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:42.128895 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ea297-74d1-4d5f-a0fe-3a9f973e466c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:42.130661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ea297-74d1-4d5f-a0fe-3a9f973e466c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:42.143172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1376/2759] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:42.152072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c1183-6a83-43a8-a748-a90ac19aa2fe admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:42.170900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c1183-6a83-43a8-a748-a90ac19aa2fe admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:42.177345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b956a-198b-4b55-813b-33cb27811a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:42.178027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b956a-198b-4b55-813b-33cb27811a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:42.178304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b956a-198b-4b55-813b-33cb27811a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:42.178604 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b956a-198b-4b55-813b-33cb27811a33 admin admin] PATH_INFO: `/v3/projects/c0ba136124fb49f1b64f1232fc0d3811/users/afbea107ec7747ceb4ee07fe20e7a468/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:42.180549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b956a-198b-4b55-813b-33cb27811a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ba136124fb49f1b64f1232fc0d3811, user_id=afbea107ec7747ceb4ee07fe20e7a468, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:42.184034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b956a-198b-4b55-813b-33cb27811a33 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:42.197204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1384/2760] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:42 2022] PUT /identity/v3/projects/c0ba136124fb49f1b64f1232fc0d3811/users/afbea107ec7747ceb4ee07fe20e7a468/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:42.267625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb24382-b292-459d-b877-160c064ceae7 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:42.267876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb24382-b292-459d-b877-160c064ceae7 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:42.268074 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb24382-b292-459d-b877-160c064ceae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:42.301468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-4bb24382-b292-459d-b877-160c064ceae7 None None] MFA Rules not processed for user `afbea107ec7747ceb4ee07fe20e7a468`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:42.302841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-4bb24382-b292-459d-b877-160c064ceae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:42.383901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1377/2761] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:42.396762 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b956a-198b-4b55-813b-33cb27811a33 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:42.398592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-7d0b956a-198b-4b55-813b-33cb27811a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:42.466779 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b956a-198b-4b55-813b-33cb27811a33 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:42.471057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbbb74-8a00-4601-974f-35cebe8b36b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:42.471722 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbbb74-8a00-4601-974f-35cebe8b36b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:42.472003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbbb74-8a00-4601-974f-35cebe8b36b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:42.472278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbbb74-8a00-4601-974f-35cebe8b36b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:42.487724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbbb74-8a00-4601-974f-35cebe8b36b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:42.489181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbbb74-8a00-4601-974f-35cebe8b36b1 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:42.499810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1385/2762] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:43.182745 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb24382-b292-459d-b877-160c064ceae7 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:43.274165 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb24382-b292-459d-b877-160c064ceae7 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:43.280114 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:43.280737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:43.281083 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:43.281314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] PATH_INFO: `/v3/users/752eeab04c114fc5a87e53f502086dd0` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:43.282480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=752eeab04c114fc5a87e53f502086dd0)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:43.284432 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:43.318890 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:43.341918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '752eeab04c114fc5a87e53f502086dd0'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:43.352606 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '752eeab04c114fc5a87e53f502086dd0'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:43.354138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '752eeab04c114fc5a87e53f502086dd0'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:43.354851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '752eeab04c114fc5a87e53f502086dd0'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:43.373750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '752eeab04c114fc5a87e53f502086dd0'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:43.388411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1378/2763] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:26:43 2022] DELETE /identity/v3/users/752eeab04c114fc5a87e53f502086dd0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:43.393493 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a6b95-f215-4674-a626-2d89cc3d1137 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:43.394003 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a6b95-f215-4674-a626-2d89cc3d1137 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:43.394112 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a6b95-f215-4674-a626-2d89cc3d1137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:43.403021 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:43.416733 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a98a6b95-f215-4674-a626-2d89cc3d1137 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:43.418039 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a98a6b95-f215-4674-a626-2d89cc3d1137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:43.488668 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1386/2764] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:43.500462 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:43.501957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:43.606386 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5259d-e4e9-4d5a-8edd-f432bbca38c5 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:43.610847 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29f99d-fdb4-4ce7-8520-11fe013a3171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:43.611616 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29f99d-fdb4-4ce7-8520-11fe013a3171 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:43.611861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29f99d-fdb4-4ce7-8520-11fe013a3171 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:43.612136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29f99d-fdb4-4ce7-8520-11fe013a3171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:43.624152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29f99d-fdb4-4ce7-8520-11fe013a3171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:43.627009 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29f99d-fdb4-4ce7-8520-11fe013a3171 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:43.647172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1379/2765] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:43.990341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a6b95-f215-4674-a626-2d89cc3d1137 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:43.992803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a98a6b95-f215-4674-a626-2d89cc3d1137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:44.016203 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a6b95-f215-4674-a626-2d89cc3d1137 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:44.021916 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:44.022719 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:44.022933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:44.023283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] PATH_INFO: `/v3/projects/10dea22ba13d4bdeaedce5af9280597b` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:44.024462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10dea22ba13d4bdeaedce5af9280597b)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:44.026451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:44.092745 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10dea22ba13d4bdeaedce5af9280597b', 'request_id': 'req-9f8a9108-301f-47c0-a7f1-17c23b60d818'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.101994 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10dea22ba13d4bdeaedce5af9280597b', 'request_id': 'req-9f8a9108-301f-47c0-a7f1-17c23b60d818'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.112341 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10dea22ba13d4bdeaedce5af9280597b', 'request_id': 'req-9f8a9108-301f-47c0-a7f1-17c23b60d818'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.130410 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] The token cache is being invalidate because project 10dea22ba13d4bdeaedce5af9280597b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:26:44.130716 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.131468 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.132908 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1387/2766] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:26:43 2022] DELETE /identity/v3/projects/10dea22ba13d4bdeaedce5af9280597b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:44.138177 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29f99d-fdb4-4ce7-8520-11fe013a3171 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:44.150717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29f99d-fdb4-4ce7-8520-11fe013a3171 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:44.212524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09caaec-edb8-48d7-b02f-8ad87c30a263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:44.213216 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09caaec-edb8-48d7-b02f-8ad87c30a263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:44.213537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09caaec-edb8-48d7-b02f-8ad87c30a263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:44.213850 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09caaec-edb8-48d7-b02f-8ad87c30a263 admin admin] PATH_INFO: `/v3/users/afbea107ec7747ceb4ee07fe20e7a468` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:44.215228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09caaec-edb8-48d7-b02f-8ad87c30a263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afbea107ec7747ceb4ee07fe20e7a468)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:44.217638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09caaec-edb8-48d7-b02f-8ad87c30a263 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:44.274489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f09caaec-edb8-48d7-b02f-8ad87c30a263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afbea107ec7747ceb4ee07fe20e7a468'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.277352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:44.282935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f09caaec-edb8-48d7-b02f-8ad87c30a263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afbea107ec7747ceb4ee07fe20e7a468'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.283998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f09caaec-edb8-48d7-b02f-8ad87c30a263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afbea107ec7747ceb4ee07fe20e7a468'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.284700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f09caaec-edb8-48d7-b02f-8ad87c30a263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afbea107ec7747ceb4ee07fe20e7a468'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.298563 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-f09caaec-edb8-48d7-b02f-8ad87c30a263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afbea107ec7747ceb4ee07fe20e7a468'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.311465 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1380/2767] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:26:44 2022] DELETE /identity/v3/users/afbea107ec7747ceb4ee07fe20e7a468 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:44.652552 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:44.655565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:44.674120 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a9108-301f-47c0-a7f1-17c23b60d818 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:44.743078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4143c10-2361-46f5-b374-9ee23b197129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:44.744293 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4143c10-2361-46f5-b374-9ee23b197129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:44.744708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4143c10-2361-46f5-b374-9ee23b197129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:44.745095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4143c10-2361-46f5-b374-9ee23b197129 admin admin] PATH_INFO: `/v3/projects/c0ba136124fb49f1b64f1232fc0d3811` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:44.746489 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4143c10-2361-46f5-b374-9ee23b197129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ba136124fb49f1b64f1232fc0d3811)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:44.748643 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4143c10-2361-46f5-b374-9ee23b197129 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:44.786504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:44.804022 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c4143c10-2361-46f5-b374-9ee23b197129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ba136124fb49f1b64f1232fc0d3811', 'request_id': 'req-c4143c10-2361-46f5-b374-9ee23b197129'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.811605 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c4143c10-2361-46f5-b374-9ee23b197129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ba136124fb49f1b64f1232fc0d3811', 'request_id': 'req-c4143c10-2361-46f5-b374-9ee23b197129'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.818617 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c4143c10-2361-46f5-b374-9ee23b197129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ba136124fb49f1b64f1232fc0d3811', 'request_id': 'req-c4143c10-2361-46f5-b374-9ee23b197129'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.834306 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c4143c10-2361-46f5-b374-9ee23b197129 admin admin] The token cache is being invalidate because project c0ba136124fb49f1b64f1232fc0d3811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:26:44.834864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c4143c10-2361-46f5-b374-9ee23b197129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.835574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-c4143c10-2361-46f5-b374-9ee23b197129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:26:44.837507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1388/2768] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:26:44 2022] DELETE /identity/v3/projects/c0ba136124fb49f1b64f1232fc0d3811 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:44.844314 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39711d9e-9144-4bc0-9fdd-ffc65deb4a88 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:44.844891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39711d9e-9144-4bc0-9fdd-ffc65deb4a88 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:44.844891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39711d9e-9144-4bc0-9fdd-ffc65deb4a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:44.869677 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-39711d9e-9144-4bc0-9fdd-ffc65deb4a88 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:44.870899 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-39711d9e-9144-4bc0-9fdd-ffc65deb4a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:44.948189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1381/2769] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:44.956123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363d9d3-9103-489d-aba4-b770f34c98e3 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:44.956385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363d9d3-9103-489d-aba4-b770f34c98e3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:44.956656 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363d9d3-9103-489d-aba4-b770f34c98e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:44.977199 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a363d9d3-9103-489d-aba4-b770f34c98e3 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:44.978456 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a363d9d3-9103-489d-aba4-b770f34c98e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:44.987373 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1389/2770] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:44.992300 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39711d9e-9144-4bc0-9fdd-ffc65deb4a88 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:45.002802 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39711d9e-9144-4bc0-9fdd-ffc65deb4a88 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:45.007380 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6f43c-71de-4f2d-b80c-80ef619d3f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:45.007993 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6f43c-71de-4f2d-b80c-80ef619d3f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:45.008269 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6f43c-71de-4f2d-b80c-80ef619d3f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:45.008536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6f43c-71de-4f2d-b80c-80ef619d3f93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:45.009190 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6f43c-71de-4f2d-b80c-80ef619d3f93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:26:45.009490 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6f43c-71de-4f2d-b80c-80ef619d3f93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:45.011195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6f43c-71de-4f2d-b80c-80ef619d3f93 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:45.023632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1382/2771] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:26:45.028863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363d9d3-9103-489d-aba4-b770f34c98e3 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:45.038774 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363d9d3-9103-489d-aba4-b770f34c98e3 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:45.045236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2577e2-0d4b-44ce-91ca-c5f87d8804a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:45.045850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2577e2-0d4b-44ce-91ca-c5f87d8804a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:45.046161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2577e2-0d4b-44ce-91ca-c5f87d8804a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:45.046434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2577e2-0d4b-44ce-91ca-c5f87d8804a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:45.047072 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2577e2-0d4b-44ce-91ca-c5f87d8804a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:45.049061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2577e2-0d4b-44ce-91ca-c5f87d8804a4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:45.079720 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1390/2772] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:26:45 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:45.086527 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6f43c-71de-4f2d-b80c-80ef619d3f93 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:45.098554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6f43c-71de-4f2d-b80c-80ef619d3f93 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:45.160163 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112ec73-76bc-4812-8ec3-4a7a96138ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:45.160777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112ec73-76bc-4812-8ec3-4a7a96138ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:45.161069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112ec73-76bc-4812-8ec3-4a7a96138ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:45.161480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112ec73-76bc-4812-8ec3-4a7a96138ad3 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:45.162118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112ec73-76bc-4812-8ec3-4a7a96138ad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:45.164092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112ec73-76bc-4812-8ec3-4a7a96138ad3 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:45.186304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1383/2773] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:26:45 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:26:45.190512 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2577e2-0d4b-44ce-91ca-c5f87d8804a4 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:45.202202 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2577e2-0d4b-44ce-91ca-c5f87d8804a4 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:45.206771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59a199-b2f2-44cb-a7c3-f7fe2773e002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:45.207422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59a199-b2f2-44cb-a7c3-f7fe2773e002 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:45.207698 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59a199-b2f2-44cb-a7c3-f7fe2773e002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:45.207970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59a199-b2f2-44cb-a7c3-f7fe2773e002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:45.208611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59a199-b2f2-44cb-a7c3-f7fe2773e002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:45.210207 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59a199-b2f2-44cb-a7c3-f7fe2773e002 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:45.221285 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1391/2774] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:45.225590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112ec73-76bc-4812-8ec3-4a7a96138ad3 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:45.236701 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112ec73-76bc-4812-8ec3-4a7a96138ad3 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:45.242486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44bd9f-1bc1-45c7-a0b6-6b47b488f4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:45.243126 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44bd9f-1bc1-45c7-a0b6-6b47b488f4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:45.243435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44bd9f-1bc1-45c7-a0b6-6b47b488f4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:45.243710 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44bd9f-1bc1-45c7-a0b6-6b47b488f4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:45.244369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44bd9f-1bc1-45c7-a0b6-6b47b488f4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:45.245991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44bd9f-1bc1-45c7-a0b6-6b47b488f4d8 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:45.257231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1384/2775] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:45.262395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59a199-b2f2-44cb-a7c3-f7fe2773e002 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:45.272821 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59a199-b2f2-44cb-a7c3-f7fe2773e002 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:45.278357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73dd5f-3a30-4050-979d-f8c7745b88aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:45.279024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73dd5f-3a30-4050-979d-f8c7745b88aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:45.279340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73dd5f-3a30-4050-979d-f8c7745b88aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:45.279620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73dd5f-3a30-4050-979d-f8c7745b88aa admin admin] PATH_INFO: `/v3/projects/bc4b2b7e8dfd4d338f2ce31ce51d3f33/users/e39920c523e346c98374eee0ade8065a/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:45.294959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73dd5f-3a30-4050-979d-f8c7745b88aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4b2b7e8dfd4d338f2ce31ce51d3f33, user_id=e39920c523e346c98374eee0ade8065a, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:45.297732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73dd5f-3a30-4050-979d-f8c7745b88aa admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:45.311309 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1392/2776] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:26:45 2022] PUT /identity/v3/projects/bc4b2b7e8dfd4d338f2ce31ce51d3f33/users/e39920c523e346c98374eee0ade8065a/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:26:45.321966 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb083d-e578-43fb-b78c-49c0af38e85d None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:45.322788 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb083d-e578-43fb-b78c-49c0af38e85d None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:45.323028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb083d-e578-43fb-b78c-49c0af38e85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:45.353700 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-ccbb083d-e578-43fb-b78c-49c0af38e85d None None] MFA Rules not processed for user `e39920c523e346c98374eee0ade8065a`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:45.354963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ccbb083d-e578-43fb-b78c-49c0af38e85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:45.386375 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:45.442530 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1385/2777] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:26:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:26:45.454641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73dd5f-3a30-4050-979d-f8c7745b88aa admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:45.456382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-3d73dd5f-3a30-4050-979d-f8c7745b88aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:45.515354 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73dd5f-3a30-4050-979d-f8c7745b88aa admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:45.519045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ee26d-9861-4c13-9ba5-fb9168e0019a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:45.519694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ee26d-9861-4c13-9ba5-fb9168e0019a service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:45.519978 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ee26d-9861-4c13-9ba5-fb9168e0019a service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:45.520254 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ee26d-9861-4c13-9ba5-fb9168e0019a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:45.528974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ee26d-9861-4c13-9ba5-fb9168e0019a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:45.530440 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ee26d-9861-4c13-9ba5-fb9168e0019a service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:45.540431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1393/2778] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:55.439769 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=17 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:55.536067 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=66,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:06.564766 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb083d-e578-43fb-b78c-49c0af38e85d None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:06.632387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb083d-e578-43fb-b78c-49c0af38e85d None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:06.637483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:06.638115 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:06.638389 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:06.638657 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] PATH_INFO: `/v3/users/e39920c523e346c98374eee0ade8065a` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:06.639781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e39920c523e346c98374eee0ade8065a)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:06.641688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:06.695062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e39920c523e346c98374eee0ade8065a'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:06.703052 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e39920c523e346c98374eee0ade8065a'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:06.703851 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e39920c523e346c98374eee0ade8065a'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:06.704501 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e39920c523e346c98374eee0ade8065a'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:06.717201 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e39920c523e346c98374eee0ade8065a'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:06.729503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1386/2779] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:27:06 2022] DELETE /identity/v3/users/e39920c523e346c98374eee0ade8065a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:27:06.734761 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280b8ba-221d-4dec-96ec-00d25362b699 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:06.735304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280b8ba-221d-4dec-96ec-00d25362b699 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:06.735561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280b8ba-221d-4dec-96ec-00d25362b699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:06.757585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-4280b8ba-221d-4dec-96ec-00d25362b699 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:27:06.759000 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4280b8ba-221d-4dec-96ec-00d25362b699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:06.825359 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1394/2780] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:27:06.835041 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:06.836528 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:06.903571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9befc7-f0bf-4a5a-b13a-3d79cf19db85 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:06.907293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdefab0-3ae4-4020-879e-1043d644303d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:06.907905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdefab0-3ae4-4020-879e-1043d644303d service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:06.908191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdefab0-3ae4-4020-879e-1043d644303d service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:06.908455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdefab0-3ae4-4020-879e-1043d644303d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:06.919795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdefab0-3ae4-4020-879e-1043d644303d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:06.921257 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdefab0-3ae4-4020-879e-1043d644303d service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:06.932034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1387/2781] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:07.227013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280b8ba-221d-4dec-96ec-00d25362b699 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:07.229487 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4280b8ba-221d-4dec-96ec-00d25362b699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:07.245495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280b8ba-221d-4dec-96ec-00d25362b699 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:07.250499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131d08f-cf9a-452f-85b5-bda945bab33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:07.251256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131d08f-cf9a-452f-85b5-bda945bab33c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:07.251483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131d08f-cf9a-452f-85b5-bda945bab33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:07.251758 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131d08f-cf9a-452f-85b5-bda945bab33c admin admin] PATH_INFO: `/v3/projects/bc4b2b7e8dfd4d338f2ce31ce51d3f33` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:07.252989 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131d08f-cf9a-452f-85b5-bda945bab33c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc4b2b7e8dfd4d338f2ce31ce51d3f33)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:07.254974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131d08f-cf9a-452f-85b5-bda945bab33c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:07.323647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8131d08f-cf9a-452f-85b5-bda945bab33c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc4b2b7e8dfd4d338f2ce31ce51d3f33', 'request_id': 'req-8131d08f-cf9a-452f-85b5-bda945bab33c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:07.332218 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8131d08f-cf9a-452f-85b5-bda945bab33c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc4b2b7e8dfd4d338f2ce31ce51d3f33', 'request_id': 'req-8131d08f-cf9a-452f-85b5-bda945bab33c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:07.339419 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8131d08f-cf9a-452f-85b5-bda945bab33c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc4b2b7e8dfd4d338f2ce31ce51d3f33', 'request_id': 'req-8131d08f-cf9a-452f-85b5-bda945bab33c'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:07.354967 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8131d08f-cf9a-452f-85b5-bda945bab33c admin admin] The token cache is being invalidate because project bc4b2b7e8dfd4d338f2ce31ce51d3f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:27:07.355326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8131d08f-cf9a-452f-85b5-bda945bab33c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:07.356546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-8131d08f-cf9a-452f-85b5-bda945bab33c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:27:07.358682 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1395/2782] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:27:07 2022] DELETE /identity/v3/projects/bc4b2b7e8dfd4d338f2ce31ce51d3f33 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:27:11.032503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdefab0-3ae4-4020-879e-1043d644303d service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:11.034085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6bdefab0-3ae4-4020-879e-1043d644303d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:11.063413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:11.099924 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdefab0-3ae4-4020-879e-1043d644303d service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:11.150005 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219bce5-6a80-45e5-b388-ba41d5b326ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:11.150615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219bce5-6a80-45e5-b388-ba41d5b326ff service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:11.150890 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219bce5-6a80-45e5-b388-ba41d5b326ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:11.151182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219bce5-6a80-45e5-b388-ba41d5b326ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:11.152947 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c219bce5-6a80-45e5-b388-ba41d5b326ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:11.228121 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219bce5-6a80-45e5-b388-ba41d5b326ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:11.229536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219bce5-6a80-45e5-b388-ba41d5b326ff service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:11.241521 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1388/2783] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:17.350914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:19.423499 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe75ce-76b5-4146-9e78-192613b7f3d3 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:19.424334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe75ce-76b5-4146-9e78-192613b7f3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:19.424334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe75ce-76b5-4146-9e78-192613b7f3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:19.447268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-d2fe75ce-76b5-4146-9e78-192613b7f3d3 None None] MFA Rules not processed for user `57896ea1ff2046d282fe47e2a8191676`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:27:19.448806 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d2fe75ce-76b5-4146-9e78-192613b7f3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:19.510252 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1396/2784] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:27:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:27:19.550758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219bce5-6a80-45e5-b388-ba41d5b326ff service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:19.552238 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-c219bce5-6a80-45e5-b388-ba41d5b326ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:19.563739 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219bce5-6a80-45e5-b388-ba41d5b326ff service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:19.567451 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb646e07-8c11-4120-a2c4-fc8651bc7fa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:19.568056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb646e07-8c11-4120-a2c4-fc8651bc7fa7 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:19.568330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb646e07-8c11-4120-a2c4-fc8651bc7fa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:19.568598 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb646e07-8c11-4120-a2c4-fc8651bc7fa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:19.570339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-cb646e07-8c11-4120-a2c4-fc8651bc7fa7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:19.585217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb646e07-8c11-4120-a2c4-fc8651bc7fa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:19.586571 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb646e07-8c11-4120-a2c4-fc8651bc7fa7 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:19.598136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1389/2785] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:21.329734 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe75ce-76b5-4146-9e78-192613b7f3d3 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:21.331735 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-d2fe75ce-76b5-4146-9e78-192613b7f3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:21.425606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe75ce-76b5-4146-9e78-192613b7f3d3 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:21.431018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84edc8-1be6-470c-82c4-aa658f717dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:21.431738 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84edc8-1be6-470c-82c4-aa658f717dfb service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:21.432012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84edc8-1be6-470c-82c4-aa658f717dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:21.432290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84edc8-1be6-470c-82c4-aa658f717dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:21.435572 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-db84edc8-1be6-470c-82c4-aa658f717dfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:21.446196 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84edc8-1be6-470c-82c4-aa658f717dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:21.447863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84edc8-1be6-470c-82c4-aa658f717dfb service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:21.458580 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1397/2786] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:27:24.382269 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d079822-e66a-40a0-aec5-61799dbbdd8c None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:24.383070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d079822-e66a-40a0-aec5-61799dbbdd8c None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:24.384251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d079822-e66a-40a0-aec5-61799dbbdd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:24.409239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-3d079822-e66a-40a0-aec5-61799dbbdd8c None None] MFA Rules not processed for user `1686024a93d34902b19c520d2b2a707d`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:27:24.410473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3d079822-e66a-40a0-aec5-61799dbbdd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:24.479413 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1390/2787] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:27:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:27:31.454081 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=51 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:34.470138 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:41.813047 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84edc8-1be6-470c-82c4-aa658f717dfb service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:41.815432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-db84edc8-1be6-470c-82c4-aa658f717dfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:41.840056 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84edc8-1be6-470c-82c4-aa658f717dfb service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:41.849036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cea1a2-734d-4179-a4f8-ceb6080a5e13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:41.850023 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cea1a2-734d-4179-a4f8-ceb6080a5e13 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:41.850399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cea1a2-734d-4179-a4f8-ceb6080a5e13 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:41.850793 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cea1a2-734d-4179-a4f8-ceb6080a5e13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:41.853757 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c1cea1a2-734d-4179-a4f8-ceb6080a5e13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:41.874221 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cea1a2-734d-4179-a4f8-ceb6080a5e13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:41.876307 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cea1a2-734d-4179-a4f8-ceb6080a5e13 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:41.894539 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1398/2788] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:43.378588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d079822-e66a-40a0-aec5-61799dbbdd8c None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:43.393504 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d079822-e66a-40a0-aec5-61799dbbdd8c None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:43.394378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19bb14-ee70-4d7b-ae53-bb94bc023c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:43.395013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19bb14-ee70-4d7b-ae53-bb94bc023c98 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:43.395357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19bb14-ee70-4d7b-ae53-bb94bc023c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:43.395624 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19bb14-ee70-4d7b-ae53-bb94bc023c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:43.397552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3f19bb14-ee70-4d7b-ae53-bb94bc023c98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:43.484777 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19bb14-ee70-4d7b-ae53-bb94bc023c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:43.486153 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19bb14-ee70-4d7b-ae53-bb94bc023c98 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:43.498381 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1391/2789] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:51.883791 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=6 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:53.492242 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=30 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:28:46.481142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90308f3b-d985-4abe-a58b-ac294c56bed2 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:46.482015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90308f3b-d985-4abe-a58b-ac294c56bed2 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:46.482015 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90308f3b-d985-4abe-a58b-ac294c56bed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:46.501655 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-90308f3b-d985-4abe-a58b-ac294c56bed2 None None] MFA Rules not processed for user `1686024a93d34902b19c520d2b2a707d`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:28:46.502860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-90308f3b-d985-4abe-a58b-ac294c56bed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:28:46.509737 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1399/2790] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:28:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:28:46.831406 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19bb14-ee70-4d7b-ae53-bb94bc023c98 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:28:46.841037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19bb14-ee70-4d7b-ae53-bb94bc023c98 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:28:46.844747 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d0031-e327-4b4c-ac0e-1703703be7ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:28:46.845358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d0031-e327-4b4c-ac0e-1703703be7ce service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:46.845635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d0031-e327-4b4c-ac0e-1703703be7ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:46.845903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d0031-e327-4b4c-ac0e-1703703be7ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:46.847699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6d3d0031-e327-4b4c-ac0e-1703703be7ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:28:46.858679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d0031-e327-4b4c-ac0e-1703703be7ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:28:46.860148 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d0031-e327-4b4c-ac0e-1703703be7ce service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:28:46.868332 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1392/2791] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:28:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:28:47.190220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90308f3b-d985-4abe-a58b-ac294c56bed2 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:28:47.191748 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-90308f3b-d985-4abe-a58b-ac294c56bed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:28:47.266401 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90308f3b-d985-4abe-a58b-ac294c56bed2 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:28:47.271694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46f75d-69f7-4586-b983-0cc4a3b875bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:28:47.272304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46f75d-69f7-4586-b983-0cc4a3b875bd service placement] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:47.272586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46f75d-69f7-4586-b983-0cc4a3b875bd service placement] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:47.272867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46f75d-69f7-4586-b983-0cc4a3b875bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:47.274634 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-6b46f75d-69f7-4586-b983-0cc4a3b875bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:28:47.286619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46f75d-69f7-4586-b983-0cc4a3b875bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:28:47.288256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46f75d-69f7-4586-b983-0cc4a3b875bd service placement] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:28:47.298750 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1400/2792] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:28:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:28:56.394085 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d0031-e327-4b4c-ac0e-1703703be7ce service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:28:56.396991 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6d3d0031-e327-4b4c-ac0e-1703703be7ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:28:56.415227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d0031-e327-4b4c-ac0e-1703703be7ce service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:28:56.422058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb8f98-1937-4d7b-9063-a822cddbd826 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:28:56.423034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb8f98-1937-4d7b-9063-a822cddbd826 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:56.423034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb8f98-1937-4d7b-9063-a822cddbd826 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:56.423304 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb8f98-1937-4d7b-9063-a822cddbd826 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:56.425245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-6dcb8f98-1937-4d7b-9063-a822cddbd826 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:28:56.442410 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb8f98-1937-4d7b-9063-a822cddbd826 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:28:56.444914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb8f98-1937-4d7b-9063-a822cddbd826 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:28:56.460042 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1393/2793] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:28:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:28:57.295210 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=35 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:29:06.455426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=12 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:29:30.878559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46f75d-69f7-4586-b983-0cc4a3b875bd service placement] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:29:30.894177 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46f75d-69f7-4586-b983-0cc4a3b875bd service placement] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:29:30.901115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0abc4-c419-4812-8dcb-5a93efbc7048 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:29:30.901814 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0abc4-c419-4812-8dcb-5a93efbc7048 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:29:30.902144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0abc4-c419-4812-8dcb-5a93efbc7048 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:29:30.902559 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0abc4-c419-4812-8dcb-5a93efbc7048 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:29:30.906261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-e2a0abc4-c419-4812-8dcb-5a93efbc7048 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:29:30.921130 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0abc4-c419-4812-8dcb-5a93efbc7048 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:29:30.923009 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0abc4-c419-4812-8dcb-5a93efbc7048 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:29:30.934375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1401/2794] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:29:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:29:33.646703 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb8f98-1937-4d7b-9063-a822cddbd826 service nova] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:29:33.657299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb8f98-1937-4d7b-9063-a822cddbd826 service nova] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:29:33.661373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f4367e-a0f4-4204-a1d0-aab554f3ab96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:29:33.661980 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4367e-a0f4-4204-a1d0-aab554f3ab96 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:29:33.662261 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4367e-a0f4-4204-a1d0-aab554f3ab96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:29:33.662558 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4367e-a0f4-4204-a1d0-aab554f3ab96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:29:33.664394 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e5f4367e-a0f4-4204-a1d0-aab554f3ab96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:29:33.707342 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f4367e-a0f4-4204-a1d0-aab554f3ab96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:29:33.709013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f4367e-a0f4-4204-a1d0-aab554f3ab96 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:29:33.730683 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1394/2795] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:29:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:29:40.929546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=6 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:29:43.108853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0abc4-c419-4812-8dcb-5a93efbc7048 service nova] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:29:43.125890 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0abc4-c419-4812-8dcb-5a93efbc7048 service nova] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:29:43.131158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebf704-2d48-4905-862d-98d886cef6e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:29:43.131786 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebf704-2d48-4905-862d-98d886cef6e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:29:43.132076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebf704-2d48-4905-862d-98d886cef6e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:29:43.132379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebf704-2d48-4905-862d-98d886cef6e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:29:43.134181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bdebf704-2d48-4905-862d-98d886cef6e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:29:43.203083 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdebf704-2d48-4905-862d-98d886cef6e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:29:43.204573 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdebf704-2d48-4905-862d-98d886cef6e6 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:29:43.215033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1402/2796] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:29:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:29:43.727524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=19 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:29:46.848773 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f4367e-a0f4-4204-a1d0-aab554f3ab96 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:29:46.868913 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f4367e-a0f4-4204-a1d0-aab554f3ab96 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:29:46.875299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f199233-65f2-4132-904b-acd17f268e96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:29:46.876015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f199233-65f2-4132-904b-acd17f268e96 service placement] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:29:46.876366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f199233-65f2-4132-904b-acd17f268e96 service placement] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:29:46.876704 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f199233-65f2-4132-904b-acd17f268e96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:29:46.880399 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5f199233-65f2-4132-904b-acd17f268e96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:29:46.902054 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f199233-65f2-4132-904b-acd17f268e96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:29:46.904268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f199233-65f2-4132-904b-acd17f268e96 service placement] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:29:46.937419 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1395/2797] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:29:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:29:50.643527 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebf704-2d48-4905-862d-98d886cef6e6 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:29:50.645107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-bdebf704-2d48-4905-862d-98d886cef6e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:29:50.657619 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebf704-2d48-4905-862d-98d886cef6e6 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:29:50.662495 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:29:50.663160 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:29:50.663448 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:29:50.663727 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] PATH_INFO: `/v3/users/57896ea1ff2046d282fe47e2a8191676` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:29:50.664863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=57896ea1ff2046d282fe47e2a8191676)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:29:50.666815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:29:50.714669 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57896ea1ff2046d282fe47e2a8191676'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:50.727082 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57896ea1ff2046d282fe47e2a8191676'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:50.734761 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57896ea1ff2046d282fe47e2a8191676'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:50.735519 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57896ea1ff2046d282fe47e2a8191676'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:50.736132 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57896ea1ff2046d282fe47e2a8191676'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:50.750141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1403/2798] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:29:50 2022] DELETE /identity/v3/users/57896ea1ff2046d282fe47e2a8191676 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:29:50.923592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f199233-65f2-4132-904b-acd17f268e96 service placement] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:29:50.936562 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f199233-65f2-4132-904b-acd17f268e96 service placement] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:29:50.996265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:29:50.996848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:29:50.997227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:29:50.997411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] PATH_INFO: `/v3/projects/963b95435fe545bb9e9ff7a1c4490a05` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:29:50.998645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=963b95435fe545bb9e9ff7a1c4490a05)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:29:51.000661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:29:51.048963 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '963b95435fe545bb9e9ff7a1c4490a05', 'request_id': 'req-95c58931-9bff-4256-87ac-b81177123c0d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.056758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '963b95435fe545bb9e9ff7a1c4490a05', 'request_id': 'req-95c58931-9bff-4256-87ac-b81177123c0d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.062806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '963b95435fe545bb9e9ff7a1c4490a05', 'request_id': 'req-95c58931-9bff-4256-87ac-b81177123c0d'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.074251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] The token cache is being invalidate because project 963b95435fe545bb9e9ff7a1c4490a05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:29:51.074251 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.074892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.076480 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1396/2799] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:29:50 2022] DELETE /identity/v3/projects/963b95435fe545bb9e9ff7a1c4490a05 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:29:51.080443 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:29:51.081945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:29:51.096632 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f496f52-fa11-4717-b24b-3e79396ce9ab admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:29:51.152442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0c9d8-9551-4c9d-8383-3adb96aa4b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:29:51.153610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0c9d8-9551-4c9d-8383-3adb96aa4b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:29:51.153885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0c9d8-9551-4c9d-8383-3adb96aa4b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:29:51.154155 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0c9d8-9551-4c9d-8383-3adb96aa4b22 admin admin] PATH_INFO: `/v3/users/9b571dfc031e46e8b003859ccaa74641` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:29:51.164534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:29:51.168812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0c9d8-9551-4c9d-8383-3adb96aa4b22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b571dfc031e46e8b003859ccaa74641)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:29:51.171535 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0c9d8-9551-4c9d-8383-3adb96aa4b22 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:29:51.218471 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4cd0c9d8-9551-4c9d-8383-3adb96aa4b22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b571dfc031e46e8b003859ccaa74641'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.230538 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4cd0c9d8-9551-4c9d-8383-3adb96aa4b22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b571dfc031e46e8b003859ccaa74641'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.238595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4cd0c9d8-9551-4c9d-8383-3adb96aa4b22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b571dfc031e46e8b003859ccaa74641'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.239385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4cd0c9d8-9551-4c9d-8383-3adb96aa4b22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b571dfc031e46e8b003859ccaa74641'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.240013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4cd0c9d8-9551-4c9d-8383-3adb96aa4b22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b571dfc031e46e8b003859ccaa74641'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.254921 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1404/2800] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:29:51 2022] DELETE /identity/v3/users/9b571dfc031e46e8b003859ccaa74641 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:29:51.443596 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:29:51.445038 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:29:51.460452 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c58931-9bff-4256-87ac-b81177123c0d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:29:51.518534 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:29:51.519108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:29:51.519408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:29:51.519678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] PATH_INFO: `/v3/projects/1c95a078e8624467b76a9d3aef768a7e` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:29:51.530115 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c95a078e8624467b76a9d3aef768a7e)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:29:51.532066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:29:51.542698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:29:51.583176 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c95a078e8624467b76a9d3aef768a7e', 'request_id': 'req-556420ee-d982-4bab-8e95-e7f60f34fd6b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.591179 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c95a078e8624467b76a9d3aef768a7e', 'request_id': 'req-556420ee-d982-4bab-8e95-e7f60f34fd6b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.597689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c95a078e8624467b76a9d3aef768a7e', 'request_id': 'req-556420ee-d982-4bab-8e95-e7f60f34fd6b'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.610741 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] The token cache is being invalidate because project 1c95a078e8624467b76a9d3aef768a7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:29:51.611025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.611741 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:29:51.613317 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1397/2801] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:29:51 2022] DELETE /identity/v3/projects/1c95a078e8624467b76a9d3aef768a7e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:30:01.250694 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=28,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:30:01.607588 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:30:02.472256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaaf2e-5176-4fc2-bfcd-182e98dec904 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:02.472999 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaaf2e-5176-4fc2-bfcd-182e98dec904 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:02.472999 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaaf2e-5176-4fc2-bfcd-182e98dec904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:02.493939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-c1aaaf2e-5176-4fc2-bfcd-182e98dec904 None None] MFA Rules not processed for user `1686024a93d34902b19c520d2b2a707d`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:30:02.495181 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c1aaaf2e-5176-4fc2-bfcd-182e98dec904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:02.559220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1405/2802] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:30:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:30:12.548724 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=25 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:30:23.264874 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:23.266440 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:23.343643 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556420ee-d982-4bab-8e95-e7f60f34fd6b admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:23.349729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab3a48-5cd0-4460-bdb3-589d40897230 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:23.350362 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab3a48-5cd0-4460-bdb3-589d40897230 service placement] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:23.350634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab3a48-5cd0-4460-bdb3-589d40897230 service placement] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:23.350907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab3a48-5cd0-4460-bdb3-589d40897230 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:23.352820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-07ab3a48-5cd0-4460-bdb3-589d40897230 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:23.370928 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab3a48-5cd0-4460-bdb3-589d40897230 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:23.373661 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab3a48-5cd0-4460-bdb3-589d40897230 service placement] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:23.390730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1398/2803] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:30:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:30:33.383782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=32 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:30:55.431970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaaf2e-5176-4fc2-bfcd-182e98dec904 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:55.433753 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-c1aaaf2e-5176-4fc2-bfcd-182e98dec904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:55.529012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaaf2e-5176-4fc2-bfcd-182e98dec904 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:55.534925 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c4035-1a5f-480d-863f-e07505f7f809 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:55.535747 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c4035-1a5f-480d-863f-e07505f7f809 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:55.536036 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c4035-1a5f-480d-863f-e07505f7f809 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:55.536312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c4035-1a5f-480d-863f-e07505f7f809 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:55.538222 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-cc4c4035-1a5f-480d-863f-e07505f7f809 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:55.551333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c4035-1a5f-480d-863f-e07505f7f809 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:55.553031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c4035-1a5f-480d-863f-e07505f7f809 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:55.567706 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1406/2804] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:30:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:31:05.559240 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=26 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:15.418634 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab3a48-5cd0-4460-bdb3-589d40897230 service placement] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:15.420062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-07ab3a48-5cd0-4460-bdb3-589d40897230 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:15.475970 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab3a48-5cd0-4460-bdb3-589d40897230 service placement] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:15.479679 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dfd02-eec8-42fb-a3f5-9950da1f46e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:15.480232 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dfd02-eec8-42fb-a3f5-9950da1f46e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:15.480508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dfd02-eec8-42fb-a3f5-9950da1f46e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:15.480776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dfd02-eec8-42fb-a3f5-9950da1f46e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:15.482748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-122dfd02-eec8-42fb-a3f5-9950da1f46e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:15.544602 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dfd02-eec8-42fb-a3f5-9950da1f46e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:15.546078 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dfd02-eec8-42fb-a3f5-9950da1f46e2 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:15.555670 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1399/2805] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:31:19.005549 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c4035-1a5f-480d-863f-e07505f7f809 service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:19.007477 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-cc4c4035-1a5f-480d-863f-e07505f7f809 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:19.022304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c4035-1a5f-480d-863f-e07505f7f809 service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:19.028981 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095873ea-3c12-4acc-b7ea-51fbf71def3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:19.029721 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095873ea-3c12-4acc-b7ea-51fbf71def3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:19.030008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095873ea-3c12-4acc-b7ea-51fbf71def3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:19.030284 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095873ea-3c12-4acc-b7ea-51fbf71def3d admin admin] PATH_INFO: `/v3/users/1686024a93d34902b19c520d2b2a707d` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:19.031600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095873ea-3c12-4acc-b7ea-51fbf71def3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1686024a93d34902b19c520d2b2a707d)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:19.033648 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095873ea-3c12-4acc-b7ea-51fbf71def3d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:19.098055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-095873ea-3c12-4acc-b7ea-51fbf71def3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1686024a93d34902b19c520d2b2a707d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:19.110363 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-095873ea-3c12-4acc-b7ea-51fbf71def3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1686024a93d34902b19c520d2b2a707d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:19.117744 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-095873ea-3c12-4acc-b7ea-51fbf71def3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1686024a93d34902b19c520d2b2a707d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:19.118518 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-095873ea-3c12-4acc-b7ea-51fbf71def3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1686024a93d34902b19c520d2b2a707d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:19.119195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-095873ea-3c12-4acc-b7ea-51fbf71def3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1686024a93d34902b19c520d2b2a707d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:19.132475 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1407/2806] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:31:19 2022] DELETE /identity/v3/users/1686024a93d34902b19c520d2b2a707d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:19.312189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dfd02-eec8-42fb-a3f5-9950da1f46e2 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:19.326154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dfd02-eec8-42fb-a3f5-9950da1f46e2 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:19.398054 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:19.398931 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:19.399447 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:19.399935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c admin admin] PATH_INFO: `/v3/projects/9a188483dd484db4962d038428cba54d` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:19.402116 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a188483dd484db4962d038428cba54d)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:19.404086 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:19.458178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a188483dd484db4962d038428cba54d', 'request_id': 'req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:19.462317 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:19.475698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a188483dd484db4962d038428cba54d', 'request_id': 'req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:19.483454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a188483dd484db4962d038428cba54d', 'request_id': 'req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:19.502209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c admin admin] The token cache is being invalidate because project 9a188483dd484db4962d038428cba54d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:31:19.502209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:19.502209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-d8e4d640-c7d2-4a5c-9cd9-f6a184678f9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:19.503392 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1400/2807] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:31:19 2022] DELETE /identity/v3/projects/9a188483dd484db4962d038428cba54d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:19.508529 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f2d63-dafa-44cc-b7c5-4a186b2973f5 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:19.509273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f2d63-dafa-44cc-b7c5-4a186b2973f5 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:19.509273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f2d63-dafa-44cc-b7c5-4a186b2973f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:19.532883 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-9a8f2d63-dafa-44cc-b7c5-4a186b2973f5 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:19.534107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9a8f2d63-dafa-44cc-b7c5-4a186b2973f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:19.606074 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1408/2808] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:31:19.614403 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809588be-4c08-4fa4-ad63-2b982e7a8b62 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:19.614673 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809588be-4c08-4fa4-ad63-2b982e7a8b62 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:19.614935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809588be-4c08-4fa4-ad63-2b982e7a8b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:19.636629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-809588be-4c08-4fa4-ad63-2b982e7a8b62 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:19.637856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-809588be-4c08-4fa4-ad63-2b982e7a8b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:19.646096 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1401/2809] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:31:19.650592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f2d63-dafa-44cc-b7c5-4a186b2973f5 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:19.661835 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f2d63-dafa-44cc-b7c5-4a186b2973f5 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:19.666570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeea126-6ca5-439c-a89c-bec3be4b3fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:19.667290 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeea126-6ca5-439c-a89c-bec3be4b3fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:19.667574 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeea126-6ca5-439c-a89c-bec3be4b3fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:19.667847 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeea126-6ca5-439c-a89c-bec3be4b3fb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:19.668498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeea126-6ca5-439c-a89c-bec3be4b3fb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:31:19.668803 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeea126-6ca5-439c-a89c-bec3be4b3fb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:19.670462 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeea126-6ca5-439c-a89c-bec3be4b3fb0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:19.683635 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1409/2810] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:31:19.687976 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809588be-4c08-4fa4-ad63-2b982e7a8b62 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:19.697583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809588be-4c08-4fa4-ad63-2b982e7a8b62 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:19.703542 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19471dc-1b70-4459-90ce-8ce929ff04c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:19.704149 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19471dc-1b70-4459-90ce-8ce929ff04c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:19.704420 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19471dc-1b70-4459-90ce-8ce929ff04c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:19.704725 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19471dc-1b70-4459-90ce-8ce929ff04c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:19.705293 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19471dc-1b70-4459-90ce-8ce929ff04c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:19.707212 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19471dc-1b70-4459-90ce-8ce929ff04c0 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:19.736631 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1402/2811] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:31:19 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:31:19.740699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeea126-6ca5-439c-a89c-bec3be4b3fb0 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:19.751998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeea126-6ca5-439c-a89c-bec3be4b3fb0 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:19.815445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67babd78-d671-4dbc-8ee6-895bb614b558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:19.816071 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67babd78-d671-4dbc-8ee6-895bb614b558 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:19.816356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67babd78-d671-4dbc-8ee6-895bb614b558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:19.817261 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67babd78-d671-4dbc-8ee6-895bb614b558 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:19.818098 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67babd78-d671-4dbc-8ee6-895bb614b558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:19.820071 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67babd78-d671-4dbc-8ee6-895bb614b558 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:19.828580 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:19.847158 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1410/2812] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:31:19 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:31:19.848467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19471dc-1b70-4459-90ce-8ce929ff04c0 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:19.859900 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19471dc-1b70-4459-90ce-8ce929ff04c0 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:19.865004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c8e0b-4e63-43cd-8954-8b7059abc9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:19.865647 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c8e0b-4e63-43cd-8954-8b7059abc9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:19.865920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c8e0b-4e63-43cd-8954-8b7059abc9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:19.866186 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c8e0b-4e63-43cd-8954-8b7059abc9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:19.867200 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c8e0b-4e63-43cd-8954-8b7059abc9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:19.868503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c8e0b-4e63-43cd-8954-8b7059abc9c1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:19.880043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1403/2813] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:31:19.884024 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67babd78-d671-4dbc-8ee6-895bb614b558 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:19.896271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67babd78-d671-4dbc-8ee6-895bb614b558 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:19.901148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4a4d0-b78d-4123-bb10-297bc9298af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:19.901761 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4a4d0-b78d-4123-bb10-297bc9298af9 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:19.902672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4a4d0-b78d-4123-bb10-297bc9298af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:19.902672 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4a4d0-b78d-4123-bb10-297bc9298af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:19.902965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4a4d0-b78d-4123-bb10-297bc9298af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:19.904606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4a4d0-b78d-4123-bb10-297bc9298af9 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:19.915992 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1411/2814] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:31:19.920742 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c8e0b-4e63-43cd-8954-8b7059abc9c1 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:19.931175 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c8e0b-4e63-43cd-8954-8b7059abc9c1 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:19.936825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53f115-48d5-4c91-89f6-ce1500454967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:19.937536 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53f115-48d5-4c91-89f6-ce1500454967 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:19.937822 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53f115-48d5-4c91-89f6-ce1500454967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:19.938094 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53f115-48d5-4c91-89f6-ce1500454967 admin admin] PATH_INFO: `/v3/projects/1443ef4f8c5d4ad6ba64eeb812882f20/users/b3e7f8f93d744a29bf085b5f5ab476cb/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:19.952880 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53f115-48d5-4c91-89f6-ce1500454967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1443ef4f8c5d4ad6ba64eeb812882f20, user_id=b3e7f8f93d744a29bf085b5f5ab476cb, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:19.955594 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53f115-48d5-4c91-89f6-ce1500454967 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:19.969764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1404/2815] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:31:19 2022] PUT /identity/v3/projects/1443ef4f8c5d4ad6ba64eeb812882f20/users/b3e7f8f93d744a29bf085b5f5ab476cb/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:19.979379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a8b80-4497-4b3d-ab23-cd0606aac67d None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:19.982078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a8b80-4497-4b3d-ab23-cd0606aac67d None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:19.982078 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a8b80-4497-4b3d-ab23-cd0606aac67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:20.013148 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-027a8b80-4497-4b3d-ab23-cd0606aac67d None None] MFA Rules not processed for user `b3e7f8f93d744a29bf085b5f5ab476cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:20.014304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-027a8b80-4497-4b3d-ab23-cd0606aac67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:20.084112 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1412/2816] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:31:20.096566 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53f115-48d5-4c91-89f6-ce1500454967 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:20.097208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ce53f115-48d5-4c91-89f6-ce1500454967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:20.178954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53f115-48d5-4c91-89f6-ce1500454967 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:20.185409 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f477852-9f60-4c15-affa-92560e0f7e65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:20.186046 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f477852-9f60-4c15-affa-92560e0f7e65 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:20.186328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f477852-9f60-4c15-affa-92560e0f7e65 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:20.186604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f477852-9f60-4c15-affa-92560e0f7e65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:20.196470 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f477852-9f60-4c15-affa-92560e0f7e65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:20.199250 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f477852-9f60-4c15-affa-92560e0f7e65 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:20.212640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1405/2817] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:31:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:31:22.911129 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a8b80-4497-4b3d-ab23-cd0606aac67d None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:22.992818 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a8b80-4497-4b3d-ab23-cd0606aac67d None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:22.998380 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:22.999052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:22.999345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:22.999705 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] PATH_INFO: `/v3/users/b3e7f8f93d744a29bf085b5f5ab476cb` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:23.000977 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e7f8f93d744a29bf085b5f5ab476cb)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:23.002990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:23.060600 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e7f8f93d744a29bf085b5f5ab476cb'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:23.075522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e7f8f93d744a29bf085b5f5ab476cb'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:23.082554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:23.088536 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e7f8f93d744a29bf085b5f5ab476cb'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:23.089860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e7f8f93d744a29bf085b5f5ab476cb'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:23.090702 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e7f8f93d744a29bf085b5f5ab476cb'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:23.114115 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1413/2818] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:31:22 2022] DELETE /identity/v3/users/b3e7f8f93d744a29bf085b5f5ab476cb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:23.119237 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e69a8-9d0c-4424-8a0f-a3daf63e8de5 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:23.119826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e69a8-9d0c-4424-8a0f-a3daf63e8de5 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:23.119826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e69a8-9d0c-4424-8a0f-a3daf63e8de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:23.140979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-719e69a8-9d0c-4424-8a0f-a3daf63e8de5 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:23.142056 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-719e69a8-9d0c-4424-8a0f-a3daf63e8de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:23.207828 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1406/2819] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:31:23.219289 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:23.220532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:23.282686 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b914e-2729-42db-a7d2-fd91addec3d7 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:23.286544 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27e3c-a714-4b74-944f-e47eb79903d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:23.287201 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27e3c-a714-4b74-944f-e47eb79903d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:23.287487 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27e3c-a714-4b74-944f-e47eb79903d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:23.287765 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27e3c-a714-4b74-944f-e47eb79903d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:23.298542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27e3c-a714-4b74-944f-e47eb79903d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:23.300220 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27e3c-a714-4b74-944f-e47eb79903d7 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:23.311500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1414/2820] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:31:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:31:23.621922 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e69a8-9d0c-4424-8a0f-a3daf63e8de5 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:23.624395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-719e69a8-9d0c-4424-8a0f-a3daf63e8de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:23.636275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:23.645336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e69a8-9d0c-4424-8a0f-a3daf63e8de5 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:23.650925 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c728f8-c988-4901-be56-735b255328b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:23.652921 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c728f8-c988-4901-be56-735b255328b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:23.653332 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c728f8-c988-4901-be56-735b255328b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:23.657021 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c728f8-c988-4901-be56-735b255328b4 admin admin] PATH_INFO: `/v3/projects/1443ef4f8c5d4ad6ba64eeb812882f20` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:23.658990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c728f8-c988-4901-be56-735b255328b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1443ef4f8c5d4ad6ba64eeb812882f20)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:23.661230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c728f8-c988-4901-be56-735b255328b4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:23.712815 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c9c728f8-c988-4901-be56-735b255328b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1443ef4f8c5d4ad6ba64eeb812882f20', 'request_id': 'req-c9c728f8-c988-4901-be56-735b255328b4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:23.721733 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c9c728f8-c988-4901-be56-735b255328b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1443ef4f8c5d4ad6ba64eeb812882f20', 'request_id': 'req-c9c728f8-c988-4901-be56-735b255328b4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:23.727528 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c9c728f8-c988-4901-be56-735b255328b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1443ef4f8c5d4ad6ba64eeb812882f20', 'request_id': 'req-c9c728f8-c988-4901-be56-735b255328b4'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:23.739446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c9c728f8-c988-4901-be56-735b255328b4 admin admin] The token cache is being invalidate because project 1443ef4f8c5d4ad6ba64eeb812882f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:31:23.739734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c9c728f8-c988-4901-be56-735b255328b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:23.740398 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-c9c728f8-c988-4901-be56-735b255328b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:23.741894 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1407/2821] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:31:23 2022] DELETE /identity/v3/projects/1443ef4f8c5d4ad6ba64eeb812882f20 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:23.746852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9bc7d-7941-4fe7-9360-e3860001ee1e None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:23.747171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9bc7d-7941-4fe7-9360-e3860001ee1e None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:23.747449 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9bc7d-7941-4fe7-9360-e3860001ee1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:23.768940 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f6e9bc7d-7941-4fe7-9360-e3860001ee1e None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:23.770171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f6e9bc7d-7941-4fe7-9360-e3860001ee1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:23.841062 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1415/2822] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:31:23.850364 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760104ed-dedc-497a-b732-5db9d6244288 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:23.850660 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760104ed-dedc-497a-b732-5db9d6244288 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:23.850930 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760104ed-dedc-497a-b732-5db9d6244288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:23.872523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-760104ed-dedc-497a-b732-5db9d6244288 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:23.873748 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-760104ed-dedc-497a-b732-5db9d6244288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:23.882316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1408/2823] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:31:23.888113 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9bc7d-7941-4fe7-9360-e3860001ee1e None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:23.900989 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9bc7d-7941-4fe7-9360-e3860001ee1e None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:23.906375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d7ad0-57f3-4ca9-b64c-0890578d4306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:23.907027 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d7ad0-57f3-4ca9-b64c-0890578d4306 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:23.907337 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d7ad0-57f3-4ca9-b64c-0890578d4306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:23.907661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d7ad0-57f3-4ca9-b64c-0890578d4306 admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:23.908275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d7ad0-57f3-4ca9-b64c-0890578d4306 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:31:23.908582 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d7ad0-57f3-4ca9-b64c-0890578d4306 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:23.910191 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d7ad0-57f3-4ca9-b64c-0890578d4306 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:23.923126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1416/2824] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:31:23.927853 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760104ed-dedc-497a-b732-5db9d6244288 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:23.942487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760104ed-dedc-497a-b732-5db9d6244288 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:23.947640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92e6d9-c04b-4b89-8a82-dddbf9682925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:23.948957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92e6d9-c04b-4b89-8a82-dddbf9682925 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:23.949217 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92e6d9-c04b-4b89-8a82-dddbf9682925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:23.949489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92e6d9-c04b-4b89-8a82-dddbf9682925 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:23.950132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92e6d9-c04b-4b89-8a82-dddbf9682925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:23.952161 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92e6d9-c04b-4b89-8a82-dddbf9682925 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:23.981735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1409/2825] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:31:23 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:31:23.986213 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d7ad0-57f3-4ca9-b64c-0890578d4306 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:23.998298 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d7ad0-57f3-4ca9-b64c-0890578d4306 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:24.067007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d0143-0fb0-4879-9846-9b4bbdf06e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:24.067677 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d0143-0fb0-4879-9846-9b4bbdf06e5a admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:24.067960 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d0143-0fb0-4879-9846-9b4bbdf06e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:24.068238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d0143-0fb0-4879-9846-9b4bbdf06e5a admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:24.068974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d0143-0fb0-4879-9846-9b4bbdf06e5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:24.070926 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d0143-0fb0-4879-9846-9b4bbdf06e5a admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:24.096200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1417/2826] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:31:23 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:31:24.101109 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92e6d9-c04b-4b89-8a82-dddbf9682925 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:24.112422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92e6d9-c04b-4b89-8a82-dddbf9682925 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:24.117295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a63f7b-1356-483a-8b57-3047c5b18fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:24.117914 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a63f7b-1356-483a-8b57-3047c5b18fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:24.118191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a63f7b-1356-483a-8b57-3047c5b18fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:24.118462 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a63f7b-1356-483a-8b57-3047c5b18fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:24.119166 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a63f7b-1356-483a-8b57-3047c5b18fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:24.121287 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a63f7b-1356-483a-8b57-3047c5b18fd1 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:24.138127 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1410/2827] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:31:24.142445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d0143-0fb0-4879-9846-9b4bbdf06e5a admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:24.153313 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d0143-0fb0-4879-9846-9b4bbdf06e5a admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:24.158205 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11588c-53be-4005-8e29-b67ca4e21c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:24.158810 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11588c-53be-4005-8e29-b67ca4e21c81 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:24.159088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11588c-53be-4005-8e29-b67ca4e21c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:24.159498 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11588c-53be-4005-8e29-b67ca4e21c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:24.160268 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11588c-53be-4005-8e29-b67ca4e21c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:24.161653 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11588c-53be-4005-8e29-b67ca4e21c81 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:24.163161 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:24.176965 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1418/2828] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:31:24.180658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a63f7b-1356-483a-8b57-3047c5b18fd1 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:24.191998 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a63f7b-1356-483a-8b57-3047c5b18fd1 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:24.196965 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597917ab-90bd-4133-aa48-cfe212c17465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:24.197639 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597917ab-90bd-4133-aa48-cfe212c17465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:24.197920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597917ab-90bd-4133-aa48-cfe212c17465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:24.198806 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597917ab-90bd-4133-aa48-cfe212c17465 admin admin] PATH_INFO: `/v3/projects/4f6d82f584be430381be39505fe455ce/users/ef451d86e2b14bf1ae377dc7d0b5090f/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:24.214352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597917ab-90bd-4133-aa48-cfe212c17465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f6d82f584be430381be39505fe455ce, user_id=ef451d86e2b14bf1ae377dc7d0b5090f, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:24.217082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597917ab-90bd-4133-aa48-cfe212c17465 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:24.230625 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1411/2829] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:31:24 2022] PUT /identity/v3/projects/4f6d82f584be430381be39505fe455ce/users/ef451d86e2b14bf1ae377dc7d0b5090f/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:24.241412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fd3df-3513-4e69-9dcd-cf3a42aba9be None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:24.241713 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fd3df-3513-4e69-9dcd-cf3a42aba9be None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:24.242049 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fd3df-3513-4e69-9dcd-cf3a42aba9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:24.271535 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-213fd3df-3513-4e69-9dcd-cf3a42aba9be None None] MFA Rules not processed for user `ef451d86e2b14bf1ae377dc7d0b5090f`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:24.272781 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-213fd3df-3513-4e69-9dcd-cf3a42aba9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:24.338500 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1419/2830] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:31:24.349742 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597917ab-90bd-4133-aa48-cfe212c17465 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:24.351464 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-597917ab-90bd-4133-aa48-cfe212c17465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:24.412296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597917ab-90bd-4133-aa48-cfe212c17465 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:24.416004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb4e1b-77cd-45cc-90c5-2758ef176c27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:24.416621 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb4e1b-77cd-45cc-90c5-2758ef176c27 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:24.417466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb4e1b-77cd-45cc-90c5-2758ef176c27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:24.417738 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb4e1b-77cd-45cc-90c5-2758ef176c27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:24.427046 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb4e1b-77cd-45cc-90c5-2758ef176c27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:24.428618 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb4e1b-77cd-45cc-90c5-2758ef176c27 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:24.438383 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1412/2831] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:31:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:31:24.459922 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fd3df-3513-4e69-9dcd-cf3a42aba9be None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:24.523277 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fd3df-3513-4e69-9dcd-cf3a42aba9be None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:24.527917 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:24.528548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:24.528831 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:24.529108 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] PATH_INFO: `/v3/users/ef451d86e2b14bf1ae377dc7d0b5090f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:24.530246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef451d86e2b14bf1ae377dc7d0b5090f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:24.532371 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:24.578198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef451d86e2b14bf1ae377dc7d0b5090f'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:24.590231 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef451d86e2b14bf1ae377dc7d0b5090f'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:24.597561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef451d86e2b14bf1ae377dc7d0b5090f'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:24.598275 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef451d86e2b14bf1ae377dc7d0b5090f'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:24.598886 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef451d86e2b14bf1ae377dc7d0b5090f'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:24.612323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1420/2832] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:31:24 2022] DELETE /identity/v3/users/ef451d86e2b14bf1ae377dc7d0b5090f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:24.616862 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed5e9f-6133-4173-8234-84c3f1beee10 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:24.617195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed5e9f-6133-4173-8234-84c3f1beee10 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:24.617463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed5e9f-6133-4173-8234-84c3f1beee10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:24.638395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-90ed5e9f-6133-4173-8234-84c3f1beee10 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:24.639635 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-90ed5e9f-6133-4173-8234-84c3f1beee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:24.658256 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:24.700040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1413/2833] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:31:24.712989 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:24.715163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:24.722374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:24.777466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b04132-bcd8-45fa-8572-1a70d5f0e58f admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:24.781071 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71feb9f9-bd18-40a9-aef3-37e91f0016aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:24.781684 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feb9f9-bd18-40a9-aef3-37e91f0016aa service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:24.782025 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feb9f9-bd18-40a9-aef3-37e91f0016aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:24.782299 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feb9f9-bd18-40a9-aef3-37e91f0016aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:24.791998 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71feb9f9-bd18-40a9-aef3-37e91f0016aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:24.793932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71feb9f9-bd18-40a9-aef3-37e91f0016aa service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:24.804057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1421/2834] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:31:25.096489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed5e9f-6133-4173-8234-84c3f1beee10 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:25.099935 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-90ed5e9f-6133-4173-8234-84c3f1beee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:25.116169 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed5e9f-6133-4173-8234-84c3f1beee10 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:25.122103 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556d843-fe94-4650-9219-c7d424ccd343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:25.122768 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556d843-fe94-4650-9219-c7d424ccd343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:25.123013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556d843-fe94-4650-9219-c7d424ccd343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:25.123305 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556d843-fe94-4650-9219-c7d424ccd343 admin admin] PATH_INFO: `/v3/projects/4f6d82f584be430381be39505fe455ce` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:25.124979 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556d843-fe94-4650-9219-c7d424ccd343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f6d82f584be430381be39505fe455ce)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:25.126351 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556d843-fe94-4650-9219-c7d424ccd343 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:25.174262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5556d843-fe94-4650-9219-c7d424ccd343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f6d82f584be430381be39505fe455ce', 'request_id': 'req-5556d843-fe94-4650-9219-c7d424ccd343'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:25.188265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5556d843-fe94-4650-9219-c7d424ccd343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f6d82f584be430381be39505fe455ce', 'request_id': 'req-5556d843-fe94-4650-9219-c7d424ccd343'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:25.194978 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5556d843-fe94-4650-9219-c7d424ccd343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f6d82f584be430381be39505fe455ce', 'request_id': 'req-5556d843-fe94-4650-9219-c7d424ccd343'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:25.213020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5556d843-fe94-4650-9219-c7d424ccd343 admin admin] The token cache is being invalidate because project 4f6d82f584be430381be39505fe455ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81795) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:31:25.213296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5556d843-fe94-4650-9219-c7d424ccd343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:25.213978 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5556d843-fe94-4650-9219-c7d424ccd343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:25.215543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1414/2835] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:31:25 2022] DELETE /identity/v3/projects/4f6d82f584be430381be39505fe455ce => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:25.222657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb9ef6-e08c-48bc-bb21-3d819ff7238b None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:25.223137 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb9ef6-e08c-48bc-bb21-3d819ff7238b None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:25.223251 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb9ef6-e08c-48bc-bb21-3d819ff7238b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:25.246007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-89bb9ef6-e08c-48bc-bb21-3d819ff7238b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:25.251031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-89bb9ef6-e08c-48bc-bb21-3d819ff7238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:25.313532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1422/2836] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:31:25.321942 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e7c20-a3b2-4a33-8d8f-4b923741ef15 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:25.322486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e7c20-a3b2-4a33-8d8f-4b923741ef15 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:25.322755 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e7c20-a3b2-4a33-8d8f-4b923741ef15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:25.343131 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-010e7c20-a3b2-4a33-8d8f-4b923741ef15 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:25.344356 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-010e7c20-a3b2-4a33-8d8f-4b923741ef15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:25.352544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1415/2837] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:31:25.357824 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb9ef6-e08c-48bc-bb21-3d819ff7238b None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:25.367952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb9ef6-e08c-48bc-bb21-3d819ff7238b None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:25.374076 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a51b3-19e8-48f1-862e-ae462c70be0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:25.374687 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a51b3-19e8-48f1-862e-ae462c70be0c admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:25.374968 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a51b3-19e8-48f1-862e-ae462c70be0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:25.375279 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a51b3-19e8-48f1-862e-ae462c70be0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:25.375936 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a51b3-19e8-48f1-862e-ae462c70be0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81796) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:31:25.376246 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a51b3-19e8-48f1-862e-ae462c70be0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:25.377885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a51b3-19e8-48f1-862e-ae462c70be0c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:25.393691 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1423/2838] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:31:25.394840 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e7c20-a3b2-4a33-8d8f-4b923741ef15 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:25.405307 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e7c20-a3b2-4a33-8d8f-4b923741ef15 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:25.410368 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef13be6-090a-4615-b495-55116e55ed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:25.410992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef13be6-090a-4615-b495-55116e55ed9e admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:25.411301 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef13be6-090a-4615-b495-55116e55ed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:25.411574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef13be6-090a-4615-b495-55116e55ed9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:25.412228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef13be6-090a-4615-b495-55116e55ed9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:25.414145 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef13be6-090a-4615-b495-55116e55ed9e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:25.444181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1416/2839] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:31:25 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:31:25.448377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a51b3-19e8-48f1-862e-ae462c70be0c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:25.458990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a51b3-19e8-48f1-862e-ae462c70be0c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:25.518442 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e1ba1-7aaa-477b-89d6-1f1dfc87c59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:25.519267 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e1ba1-7aaa-477b-89d6-1f1dfc87c59c admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:25.519552 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e1ba1-7aaa-477b-89d6-1f1dfc87c59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:25.519843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e1ba1-7aaa-477b-89d6-1f1dfc87c59c admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:25.520484 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e1ba1-7aaa-477b-89d6-1f1dfc87c59c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:25.522446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e1ba1-7aaa-477b-89d6-1f1dfc87c59c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:25.545471 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1424/2840] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:31:25 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:31:25.549665 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef13be6-090a-4615-b495-55116e55ed9e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:25.560526 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef13be6-090a-4615-b495-55116e55ed9e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:25.565358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa45be-422c-43e6-ab7c-0f901d9a9bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:25.565970 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa45be-422c-43e6-ab7c-0f901d9a9bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:25.566245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa45be-422c-43e6-ab7c-0f901d9a9bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:25.566516 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa45be-422c-43e6-ab7c-0f901d9a9bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:25.567196 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa45be-422c-43e6-ab7c-0f901d9a9bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:25.568795 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa45be-422c-43e6-ab7c-0f901d9a9bc4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:25.579295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1417/2841] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:31:25.583586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e1ba1-7aaa-477b-89d6-1f1dfc87c59c admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:25.594439 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e1ba1-7aaa-477b-89d6-1f1dfc87c59c admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:25.599522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567ec83-3e06-4a7c-9613-31dbf72125d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:25.600140 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567ec83-3e06-4a7c-9613-31dbf72125d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:25.600421 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567ec83-3e06-4a7c-9613-31dbf72125d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:25.600697 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567ec83-3e06-4a7c-9613-31dbf72125d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:25.601347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567ec83-3e06-4a7c-9613-31dbf72125d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:25.602944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567ec83-3e06-4a7c-9613-31dbf72125d1 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:25.619040 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1425/2842] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:31:25.620343 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa45be-422c-43e6-ab7c-0f901d9a9bc4 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:25.631166 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa45be-422c-43e6-ab7c-0f901d9a9bc4 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:25.636218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1b241-3a32-4ca2-993b-561cb8e7eae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:25.636901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1b241-3a32-4ca2-993b-561cb8e7eae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:25.637182 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1b241-3a32-4ca2-993b-561cb8e7eae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:25.637454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1b241-3a32-4ca2-993b-561cb8e7eae4 admin admin] PATH_INFO: `/v3/projects/6c27224f907442629923f9e48a235ed2/users/ee07deb68bbe40c9858e8e4265b66ebd/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:25.653717 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f1b241-3a32-4ca2-993b-561cb8e7eae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c27224f907442629923f9e48a235ed2, user_id=ee07deb68bbe40c9858e8e4265b66ebd, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:25.656421 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f1b241-3a32-4ca2-993b-561cb8e7eae4 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:25.671036 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1418/2843] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:31:25 2022] PUT /identity/v3/projects/6c27224f907442629923f9e48a235ed2/users/ee07deb68bbe40c9858e8e4265b66ebd/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:25.682599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9a7be-94df-4b97-be34-189f85bdddd7 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:25.682885 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9a7be-94df-4b97-be34-189f85bdddd7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:25.683200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9a7be-94df-4b97-be34-189f85bdddd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:25.712447 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:25.716737 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-24e9a7be-94df-4b97-be34-189f85bdddd7 None None] MFA Rules not processed for user `ee07deb68bbe40c9858e8e4265b66ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:25.718385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-24e9a7be-94df-4b97-be34-189f85bdddd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:25.790647 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1426/2844] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:31:25.796234 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1b241-3a32-4ca2-993b-561cb8e7eae4 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:25.852198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:25.874344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1b241-3a32-4ca2-993b-561cb8e7eae4 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:25.879902 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572d391-a0ca-4c55-a16d-380714c0338c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:25.880539 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572d391-a0ca-4c55-a16d-380714c0338c admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:25.880812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572d391-a0ca-4c55-a16d-380714c0338c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:25.881088 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572d391-a0ca-4c55-a16d-380714c0338c admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:25.881735 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572d391-a0ca-4c55-a16d-380714c0338c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:25.883713 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572d391-a0ca-4c55-a16d-380714c0338c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:25.917181 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1419/2845] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:31:25 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:31:25.921954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9a7be-94df-4b97-be34-189f85bdddd7 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:25.936088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9a7be-94df-4b97-be34-189f85bdddd7 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:26.013944 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359615c-c2a2-44ae-a59b-92a6e4f49b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:26.014624 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359615c-c2a2-44ae-a59b-92a6e4f49b72 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:26.014929 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359615c-c2a2-44ae-a59b-92a6e4f49b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:26.015236 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359615c-c2a2-44ae-a59b-92a6e4f49b72 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:26.015906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359615c-c2a2-44ae-a59b-92a6e4f49b72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:26.017875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359615c-c2a2-44ae-a59b-92a6e4f49b72 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:26.041237 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1427/2846] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:31:25 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:31:26.045473 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572d391-a0ca-4c55-a16d-380714c0338c admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:26.056628 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572d391-a0ca-4c55-a16d-380714c0338c admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:26.061699 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7a056-8762-4733-9ba8-786f28c0eecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:26.062295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7a056-8762-4733-9ba8-786f28c0eecc admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:26.062576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7a056-8762-4733-9ba8-786f28c0eecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:26.062848 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7a056-8762-4733-9ba8-786f28c0eecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:26.065523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7a056-8762-4733-9ba8-786f28c0eecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:26.065523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7a056-8762-4733-9ba8-786f28c0eecc admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:26.077797 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1420/2847] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:31:26.082610 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359615c-c2a2-44ae-a59b-92a6e4f49b72 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:26.093641 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359615c-c2a2-44ae-a59b-92a6e4f49b72 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:26.098427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9154e6-a6b5-4ae9-9e67-6757d13bbc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:26.099044 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9154e6-a6b5-4ae9-9e67-6757d13bbc0c admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:26.099356 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9154e6-a6b5-4ae9-9e67-6757d13bbc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:26.099632 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9154e6-a6b5-4ae9-9e67-6757d13bbc0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:26.100279 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9154e6-a6b5-4ae9-9e67-6757d13bbc0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:26.101868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9154e6-a6b5-4ae9-9e67-6757d13bbc0c admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:26.112954 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1428/2848] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:31:26.117243 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7a056-8762-4733-9ba8-786f28c0eecc admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:26.127568 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7a056-8762-4733-9ba8-786f28c0eecc admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:26.132342 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c3934-cd92-4521-86e3-ad5715a60bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:26.132950 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c3934-cd92-4521-86e3-ad5715a60bb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:26.133227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c3934-cd92-4521-86e3-ad5715a60bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:26.133498 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c3934-cd92-4521-86e3-ad5715a60bb8 admin admin] PATH_INFO: `/v3/projects/e88df815eb534550b6ca5446b397dfd1/users/c28c26caebe94d40a6a007dd42f26c29/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:26.148764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c3934-cd92-4521-86e3-ad5715a60bb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e88df815eb534550b6ca5446b397dfd1, user_id=c28c26caebe94d40a6a007dd42f26c29, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:26.152063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c3934-cd92-4521-86e3-ad5715a60bb8 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:26.165260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1421/2849] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:31:26 2022] PUT /identity/v3/projects/e88df815eb534550b6ca5446b397dfd1/users/c28c26caebe94d40a6a007dd42f26c29/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:26.176144 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed80c32-f2ff-47a8-9de1-601e66c58b69 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:26.176427 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed80c32-f2ff-47a8-9de1-601e66c58b69 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:26.176700 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed80c32-f2ff-47a8-9de1-601e66c58b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:26.206878 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-2ed80c32-f2ff-47a8-9de1-601e66c58b69 None None] MFA Rules not processed for user `c28c26caebe94d40a6a007dd42f26c29`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:26.208156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2ed80c32-f2ff-47a8-9de1-601e66c58b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:26.275383 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1429/2850] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:31:26.281156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c3934-cd92-4521-86e3-ad5715a60bb8 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:26.347053 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c3934-cd92-4521-86e3-ad5715a60bb8 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:26.351929 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d16ca-2f1d-41d2-a46a-6a50cc021b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:26.352550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d16ca-2f1d-41d2-a46a-6a50cc021b84 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:26.352832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d16ca-2f1d-41d2-a46a-6a50cc021b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:26.353118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d16ca-2f1d-41d2-a46a-6a50cc021b84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:26.353785 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d16ca-2f1d-41d2-a46a-6a50cc021b84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:26.356572 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d16ca-2f1d-41d2-a46a-6a50cc021b84 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:26.386750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1422/2851] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:31:26 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:31:26.390742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed80c32-f2ff-47a8-9de1-601e66c58b69 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:26.403124 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed80c32-f2ff-47a8-9de1-601e66c58b69 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:26.412101 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:26.472592 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c83830-de60-48d4-86a4-7e9ef8c18216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:26.473267 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c83830-de60-48d4-86a4-7e9ef8c18216 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:26.473994 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c83830-de60-48d4-86a4-7e9ef8c18216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:26.473994 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c83830-de60-48d4-86a4-7e9ef8c18216 admin admin] PATH_INFO: `/v3/users` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:26.480175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c83830-de60-48d4-86a4-7e9ef8c18216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:26.480175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c83830-de60-48d4-86a4-7e9ef8c18216 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:26.506088 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1430/2852] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:31:26 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:31:26.511579 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d16ca-2f1d-41d2-a46a-6a50cc021b84 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:26.523028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d16ca-2f1d-41d2-a46a-6a50cc021b84 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:26.528865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb693dd-b214-4b49-b128-18b76ac055cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:26.529495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb693dd-b214-4b49-b128-18b76ac055cc admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:26.529782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb693dd-b214-4b49-b128-18b76ac055cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:26.530059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb693dd-b214-4b49-b128-18b76ac055cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:26.530711 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb693dd-b214-4b49-b128-18b76ac055cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:26.532414 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb693dd-b214-4b49-b128-18b76ac055cc admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:26.544426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1423/2853] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:31:26.549033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c83830-de60-48d4-86a4-7e9ef8c18216 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:26.560640 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c83830-de60-48d4-86a4-7e9ef8c18216 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:26.566436 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a92e3-bc1a-4c5e-8c6d-da97ffe930a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:26.567149 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a92e3-bc1a-4c5e-8c6d-da97ffe930a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:26.567579 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a92e3-bc1a-4c5e-8c6d-da97ffe930a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:26.567715 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a92e3-bc1a-4c5e-8c6d-da97ffe930a3 admin admin] PATH_INFO: `/v3/projects/53e0e3b50198472f96a459662df5b35a/users/bff139d60e464d10b12a898d8d525304/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:26.585374 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a92e3-bc1a-4c5e-8c6d-da97ffe930a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e0e3b50198472f96a459662df5b35a, user_id=bff139d60e464d10b12a898d8d525304, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:26.588105 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a92e3-bc1a-4c5e-8c6d-da97ffe930a3 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:26.612042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1431/2854] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:31:26 2022] PUT /identity/v3/projects/53e0e3b50198472f96a459662df5b35a/users/bff139d60e464d10b12a898d8d525304/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:31:26.618270 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb693dd-b214-4b49-b128-18b76ac055cc admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:26.639361 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb693dd-b214-4b49-b128-18b76ac055cc admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:26.728344 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9df2b-6dd7-430e-af31-633aa6360b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:26.730554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9df2b-6dd7-430e-af31-633aa6360b7e admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:26.730554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9df2b-6dd7-430e-af31-633aa6360b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:26.730554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9df2b-6dd7-430e-af31-633aa6360b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:26.730554 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9df2b-6dd7-430e-af31-633aa6360b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:26.732957 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9df2b-6dd7-430e-af31-633aa6360b7e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:26.744385 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1424/2855] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:31:26.748983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a92e3-bc1a-4c5e-8c6d-da97ffe930a3 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:26.760974 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a92e3-bc1a-4c5e-8c6d-da97ffe930a3 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:26.766205 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a3520-f7ca-4af9-a32f-86e53cc058a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:26.766914 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a3520-f7ca-4af9-a32f-86e53cc058a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:26.767227 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a3520-f7ca-4af9-a32f-86e53cc058a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:26.767508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a3520-f7ca-4af9-a32f-86e53cc058a2 admin admin] PATH_INFO: `/v3/projects/53e0e3b50198472f96a459662df5b35a/users/bff139d60e464d10b12a898d8d525304/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:26.769560 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a3520-f7ca-4af9-a32f-86e53cc058a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e0e3b50198472f96a459662df5b35a, user_id=bff139d60e464d10b12a898d8d525304, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:26.772281 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a3520-f7ca-4af9-a32f-86e53cc058a2 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:26.784927 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1432/2856] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:31:26 2022] PUT /identity/v3/projects/53e0e3b50198472f96a459662df5b35a/users/bff139d60e464d10b12a898d8d525304/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:26.795369 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09834abb-8fff-492d-81e1-8eb6e4741c34 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:26.795644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09834abb-8fff-492d-81e1-8eb6e4741c34 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:26.795911 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09834abb-8fff-492d-81e1-8eb6e4741c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:26.804371 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:26.844224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-09834abb-8fff-492d-81e1-8eb6e4741c34 None None] MFA Rules not processed for user `bff139d60e464d10b12a898d8d525304`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:26.845466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-09834abb-8fff-492d-81e1-8eb6e4741c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:26.919937 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1425/2857] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:31:26.931526 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a3520-f7ca-4af9-a32f-86e53cc058a2 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:26.933929 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-eb3a3520-f7ca-4af9-a32f-86e53cc058a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:26.993633 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a3520-f7ca-4af9-a32f-86e53cc058a2 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:26.997996 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da8b40-e6f0-4eb9-8db1-8f2970b1663f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:26.998627 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da8b40-e6f0-4eb9-8db1-8f2970b1663f service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:26.999042 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da8b40-e6f0-4eb9-8db1-8f2970b1663f service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:26.999509 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da8b40-e6f0-4eb9-8db1-8f2970b1663f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:27.061034 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da8b40-e6f0-4eb9-8db1-8f2970b1663f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:27.062483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da8b40-e6f0-4eb9-8db1-8f2970b1663f service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:27.072101 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1433/2858] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:31:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:31:28.660645 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09834abb-8fff-492d-81e1-8eb6e4741c34 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:28.662140 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-09834abb-8fff-492d-81e1-8eb6e4741c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:28.686252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09834abb-8fff-492d-81e1-8eb6e4741c34 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:28.690037 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792047cf-31be-4cef-bdd0-5c5145c63c17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:28.690671 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792047cf-31be-4cef-bdd0-5c5145c63c17 service cinder] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:28.690941 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792047cf-31be-4cef-bdd0-5c5145c63c17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:28.691245 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792047cf-31be-4cef-bdd0-5c5145c63c17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:28.760117 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792047cf-31be-4cef-bdd0-5c5145c63c17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:28.763907 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792047cf-31be-4cef-bdd0-5c5145c63c17 service cinder] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:28.779811 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1426/2859] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:31:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:31:29.404975 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da8b40-e6f0-4eb9-8db1-8f2970b1663f service cinder] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:29.413983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da8b40-e6f0-4eb9-8db1-8f2970b1663f service cinder] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:29.418394 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385620d-4304-4333-9943-785bc93185e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8356f60a0f684484abc8f8d6fb6a7be6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:29.419019 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385620d-4304-4333-9943-785bc93185e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:29.419333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385620d-4304-4333-9943-785bc93185e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:29.419613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385620d-4304-4333-9943-785bc93185e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:29.430906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385620d-4304-4333-9943-785bc93185e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:29.432402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385620d-4304-4333-9943-785bc93185e8 service cinder] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:29.444615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1434/2860] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:31:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:31:33.541973 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792047cf-31be-4cef-bdd0-5c5145c63c17 service cinder] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:33.604891 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792047cf-31be-4cef-bdd0-5c5145c63c17 service cinder] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:33.609678 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:33.610319 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:33.610621 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:33.610892 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] PATH_INFO: `/v3/users/ee07deb68bbe40c9858e8e4265b66ebd` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:33.612046 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee07deb68bbe40c9858e8e4265b66ebd)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:33.613974 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:33.650705 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:33.665951 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee07deb68bbe40c9858e8e4265b66ebd'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:33.673973 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee07deb68bbe40c9858e8e4265b66ebd'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:33.674599 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee07deb68bbe40c9858e8e4265b66ebd'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:33.675228 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee07deb68bbe40c9858e8e4265b66ebd'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:33.689829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee07deb68bbe40c9858e8e4265b66ebd'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:33.702796 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1427/2861] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:31:33 2022] DELETE /identity/v3/users/ee07deb68bbe40c9858e8e4265b66ebd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:33.707354 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d175a-df70-4598-96b8-505780651847 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:33.707822 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d175a-df70-4598-96b8-505780651847 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:33.708054 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d175a-df70-4598-96b8-505780651847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:33.716929 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:33.731026 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-5e1d175a-df70-4598-96b8-505780651847 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:33.732456 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5e1d175a-df70-4598-96b8-505780651847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:33.794690 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1435/2862] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:31:33.804932 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:33.806545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:33.869068 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b85f5-11c6-4c09-aa3a-46f062c56201 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:33.872926 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b37c9c-248f-4adb-b577-90514c423e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:33.873577 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b37c9c-248f-4adb-b577-90514c423e56 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:33.873876 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b37c9c-248f-4adb-b577-90514c423e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:33.874154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b37c9c-248f-4adb-b577-90514c423e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:33.883861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b37c9c-248f-4adb-b577-90514c423e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:33.885404 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b37c9c-248f-4adb-b577-90514c423e56 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:33.895737 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1428/2863] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:31:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:31:34.197660 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d175a-df70-4598-96b8-505780651847 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:34.200135 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-5e1d175a-df70-4598-96b8-505780651847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:34.214696 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d175a-df70-4598-96b8-505780651847 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:34.220542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:34.221233 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:34.221575 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:34.221861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] PATH_INFO: `/v3/projects/6c27224f907442629923f9e48a235ed2` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:34.223534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c27224f907442629923f9e48a235ed2)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:34.227849 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:34.283514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c27224f907442629923f9e48a235ed2', 'request_id': 'req-4776599e-2bec-4134-bf5a-083f0fbaa3b3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.290476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c27224f907442629923f9e48a235ed2', 'request_id': 'req-4776599e-2bec-4134-bf5a-083f0fbaa3b3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.296557 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c27224f907442629923f9e48a235ed2', 'request_id': 'req-4776599e-2bec-4134-bf5a-083f0fbaa3b3'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.309375 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] The token cache is being invalidate because project 6c27224f907442629923f9e48a235ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:31:34.309730 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.310412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.312018 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1436/2864] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:31:34 2022] DELETE /identity/v3/projects/6c27224f907442629923f9e48a235ed2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:34.316856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b37c9c-248f-4adb-b577-90514c423e56 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:34.327651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b37c9c-248f-4adb-b577-90514c423e56 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:34.386820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:34.387508 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:34.387782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:34.388082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] PATH_INFO: `/v3/users/c28c26caebe94d40a6a007dd42f26c29` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:34.389295 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c28c26caebe94d40a6a007dd42f26c29)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:34.391276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:34.448382 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c28c26caebe94d40a6a007dd42f26c29'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.452262 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:34.456901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c28c26caebe94d40a6a007dd42f26c29'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.457654 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c28c26caebe94d40a6a007dd42f26c29'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.458610 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c28c26caebe94d40a6a007dd42f26c29'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.472223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c28c26caebe94d40a6a007dd42f26c29'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.484901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1429/2865] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:31:34 2022] DELETE /identity/v3/users/c28c26caebe94d40a6a007dd42f26c29 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:34.794545 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:34.796146 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:34.811248 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776599e-2bec-4134-bf5a-083f0fbaa3b3 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:34.872450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:34.872875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:34.873156 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:34.873429 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] PATH_INFO: `/v3/projects/e88df815eb534550b6ca5446b397dfd1` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:34.874584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e88df815eb534550b6ca5446b397dfd1)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:34.876902 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:34.917005 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:34.934204 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e88df815eb534550b6ca5446b397dfd1', 'request_id': 'req-33af6776-0f5c-4a65-b331-95ce55b200a5'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.941437 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e88df815eb534550b6ca5446b397dfd1', 'request_id': 'req-33af6776-0f5c-4a65-b331-95ce55b200a5'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.948629 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e88df815eb534550b6ca5446b397dfd1', 'request_id': 'req-33af6776-0f5c-4a65-b331-95ce55b200a5'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.964583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] The token cache is being invalidate because project e88df815eb534550b6ca5446b397dfd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:31:34.964970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.965683 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:34.967332 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1437/2866] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:31:34 2022] DELETE /identity/v3/projects/e88df815eb534550b6ca5446b397dfd1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:34.971836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:34.973308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:34.987511 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35856e91-229a-4ee1-90cb-5228191d29a5 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:35.056082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e62ba79-697b-45a6-b498-6ccd618b2b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:35.056742 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62ba79-697b-45a6-b498-6ccd618b2b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:35.057026 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62ba79-697b-45a6-b498-6ccd618b2b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:35.057296 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e62ba79-697b-45a6-b498-6ccd618b2b25 admin admin] PATH_INFO: `/v3/users/bff139d60e464d10b12a898d8d525304` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:35.058547 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e62ba79-697b-45a6-b498-6ccd618b2b25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bff139d60e464d10b12a898d8d525304)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:35.060478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e62ba79-697b-45a6-b498-6ccd618b2b25 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:35.126288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1e62ba79-697b-45a6-b498-6ccd618b2b25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bff139d60e464d10b12a898d8d525304'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:35.136653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1e62ba79-697b-45a6-b498-6ccd618b2b25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bff139d60e464d10b12a898d8d525304'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:35.137495 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1e62ba79-697b-45a6-b498-6ccd618b2b25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bff139d60e464d10b12a898d8d525304'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:35.138358 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1e62ba79-697b-45a6-b498-6ccd618b2b25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bff139d60e464d10b12a898d8d525304'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:35.154496 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-1e62ba79-697b-45a6-b498-6ccd618b2b25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bff139d60e464d10b12a898d8d525304'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:35.178195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1430/2867] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:31:34 2022] DELETE /identity/v3/users/bff139d60e464d10b12a898d8d525304 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:35.479399 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:35.481059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:35.496244 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af6776-0f5c-4a65-b331-95ce55b200a5 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:35.567785 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f34119-6b2d-4579-ae29-1f13a3e953c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:35.568450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f34119-6b2d-4579-ae29-1f13a3e953c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:35.568728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f34119-6b2d-4579-ae29-1f13a3e953c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:35.569001 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f34119-6b2d-4579-ae29-1f13a3e953c0 admin admin] PATH_INFO: `/v3/projects/53e0e3b50198472f96a459662df5b35a` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:35.570183 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f34119-6b2d-4579-ae29-1f13a3e953c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e0e3b50198472f96a459662df5b35a)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:35.572872 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f34119-6b2d-4579-ae29-1f13a3e953c0 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:35.645344 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-d7f34119-6b2d-4579-ae29-1f13a3e953c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e0e3b50198472f96a459662df5b35a', 'request_id': 'req-d7f34119-6b2d-4579-ae29-1f13a3e953c0'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:35.653388 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-d7f34119-6b2d-4579-ae29-1f13a3e953c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e0e3b50198472f96a459662df5b35a', 'request_id': 'req-d7f34119-6b2d-4579-ae29-1f13a3e953c0'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:35.661853 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-d7f34119-6b2d-4579-ae29-1f13a3e953c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e0e3b50198472f96a459662df5b35a', 'request_id': 'req-d7f34119-6b2d-4579-ae29-1f13a3e953c0'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:35.678963 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-d7f34119-6b2d-4579-ae29-1f13a3e953c0 admin admin] The token cache is being invalidate because project 53e0e3b50198472f96a459662df5b35a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:31:35.679305 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-d7f34119-6b2d-4579-ae29-1f13a3e953c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:35.679990 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-d7f34119-6b2d-4579-ae29-1f13a3e953c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:31:35.681537 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1438/2868] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:31:35 2022] DELETE /identity/v3/projects/53e0e3b50198472f96a459662df5b35a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:35.690010 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f5c73-3a1f-4d5c-bfb2-7367bdeaae12 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:35.690240 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f5c73-3a1f-4d5c-bfb2-7367bdeaae12 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:35.690475 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f5c73-3a1f-4d5c-bfb2-7367bdeaae12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:35.719166 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-088f5c73-3a1f-4d5c-bfb2-7367bdeaae12 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:35.720438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-088f5c73-3a1f-4d5c-bfb2-7367bdeaae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:35.794159 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:35.817817 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1431/2869] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:31:35.825829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2589e02-3a3d-4009-8791-bfe33e066d09 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:35.826433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2589e02-3a3d-4009-8791-bfe33e066d09 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:35.826433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2589e02-3a3d-4009-8791-bfe33e066d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:35.851187 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-a2589e02-3a3d-4009-8791-bfe33e066d09 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:35.852550 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-a2589e02-3a3d-4009-8791-bfe33e066d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:35.863013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1439/2870] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:31:35.867964 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f5c73-3a1f-4d5c-bfb2-7367bdeaae12 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:35.892396 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f5c73-3a1f-4d5c-bfb2-7367bdeaae12 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:35.901483 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ed2ae-1027-4d03-b0c7-fda74b2a662c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:35.902512 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ed2ae-1027-4d03-b0c7-fda74b2a662c admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:35.902983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ed2ae-1027-4d03-b0c7-fda74b2a662c admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:35.903544 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ed2ae-1027-4d03-b0c7-fda74b2a662c admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:35.904652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ed2ae-1027-4d03-b0c7-fda74b2a662c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:31:35.905178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ed2ae-1027-4d03-b0c7-fda74b2a662c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:35.914772 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ed2ae-1027-4d03-b0c7-fda74b2a662c admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:35.926284 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1432/2871] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:31:35.932907 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2589e02-3a3d-4009-8791-bfe33e066d09 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:35.945223 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2589e02-3a3d-4009-8791-bfe33e066d09 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:35.953606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4abe2a-5e0c-4f74-bc9e-a17856767718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:35.954740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4abe2a-5e0c-4f74-bc9e-a17856767718 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:35.955283 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4abe2a-5e0c-4f74-bc9e-a17856767718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:35.955782 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4abe2a-5e0c-4f74-bc9e-a17856767718 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:35.957064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4abe2a-5e0c-4f74-bc9e-a17856767718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:35.960837 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4abe2a-5e0c-4f74-bc9e-a17856767718 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:36.014979 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1440/2872] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:31:35 2022] POST /identity/v3/projects => generated 369 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:31:36.021023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ed2ae-1027-4d03-b0c7-fda74b2a662c admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:36.035632 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ed2ae-1027-4d03-b0c7-fda74b2a662c admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:36.130438 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff945249-4b3f-4128-aa9f-9b67bdc766c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:36.131209 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff945249-4b3f-4128-aa9f-9b67bdc766c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:36.131638 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff945249-4b3f-4128-aa9f-9b67bdc766c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:36.132033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff945249-4b3f-4128-aa9f-9b67bdc766c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:36.132812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff945249-4b3f-4128-aa9f-9b67bdc766c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:36.135004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff945249-4b3f-4128-aa9f-9b67bdc766c9 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:36.161718 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1433/2873] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:31:36 2022] POST /identity/v3/users => generated 285 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:31:36.165969 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4abe2a-5e0c-4f74-bc9e-a17856767718 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:36.178717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4abe2a-5e0c-4f74-bc9e-a17856767718 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:36.184334 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbda0de-9696-417e-a1ef-4aeae8af3cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:36.185055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbda0de-9696-417e-a1ef-4aeae8af3cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:36.185379 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbda0de-9696-417e-a1ef-4aeae8af3cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:36.185696 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbda0de-9696-417e-a1ef-4aeae8af3cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:36.186404 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbda0de-9696-417e-a1ef-4aeae8af3cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:36.188217 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbda0de-9696-417e-a1ef-4aeae8af3cd3 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:36.200919 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1441/2874] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:31:36.206136 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff945249-4b3f-4128-aa9f-9b67bdc766c9 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:36.222255 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff945249-4b3f-4128-aa9f-9b67bdc766c9 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:36.227541 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2378dd-5f21-4484-9db4-594437dcf581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:36.228188 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2378dd-5f21-4484-9db4-594437dcf581 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:36.228443 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2378dd-5f21-4484-9db4-594437dcf581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:36.228709 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2378dd-5f21-4484-9db4-594437dcf581 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:36.230061 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2378dd-5f21-4484-9db4-594437dcf581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:36.231981 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2378dd-5f21-4484-9db4-594437dcf581 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:36.244434 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1434/2875] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:31:36.249826 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbda0de-9696-417e-a1ef-4aeae8af3cd3 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:36.264708 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbda0de-9696-417e-a1ef-4aeae8af3cd3 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:36.269692 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ab457-bea4-40a9-b5d6-68cf0acb0473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:36.270355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ab457-bea4-40a9-b5d6-68cf0acb0473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:36.270654 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ab457-bea4-40a9-b5d6-68cf0acb0473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:36.270929 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ab457-bea4-40a9-b5d6-68cf0acb0473 admin admin] PATH_INFO: `/v3/projects/b14eecc2f10b455593468ccbcd52f912/users/055010641d1a4d0a9c57f2770bf2b677/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:36.275345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:36.300826 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ab457-bea4-40a9-b5d6-68cf0acb0473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b14eecc2f10b455593468ccbcd52f912, user_id=055010641d1a4d0a9c57f2770bf2b677, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:36.305012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ab457-bea4-40a9-b5d6-68cf0acb0473 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:36.327059 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1442/2876] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:31:36 2022] PUT /identity/v3/projects/b14eecc2f10b455593468ccbcd52f912/users/055010641d1a4d0a9c57f2770bf2b677/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:31:36.345328 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba36bd7-8d53-488e-9eab-245d2008d9eb None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:36.345666 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba36bd7-8d53-488e-9eab-245d2008d9eb None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:36.346015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba36bd7-8d53-488e-9eab-245d2008d9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:36.386527 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-aba36bd7-8d53-488e-9eab-245d2008d9eb None None] MFA Rules not processed for user `055010641d1a4d0a9c57f2770bf2b677`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:31:36.387776 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-aba36bd7-8d53-488e-9eab-245d2008d9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:36.462208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1435/2877] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:31:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:31:36.480566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ab457-bea4-40a9-b5d6-68cf0acb0473 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:31:36.484657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-340ab457-bea4-40a9-b5d6-68cf0acb0473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:31:36.609051 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ab457-bea4-40a9-b5d6-68cf0acb0473 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:31:36.615378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e6779-a767-4252-b094-b6e127abe1f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f82adce75b44c6baf85b6b96dd0728', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:31:36.616393 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e6779-a767-4252-b094-b6e127abe1f0 service glance] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:31:36.616874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e6779-a767-4252-b094-b6e127abe1f0 service glance] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:31:36.617345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e6779-a767-4252-b094-b6e127abe1f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:31:36.633700 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e6779-a767-4252-b094-b6e127abe1f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:31:36.636881 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e6779-a767-4252-b094-b6e127abe1f0 service glance] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:31:36.653175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1443/2878] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:31:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:31:46.458556 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:31:46.646534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:13.550917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba36bd7-8d53-488e-9eab-245d2008d9eb None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:13.618517 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba36bd7-8d53-488e-9eab-245d2008d9eb None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:13.623265 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:13.623917 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:13.624198 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:13.624475 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] PATH_INFO: `/v3/users/055010641d1a4d0a9c57f2770bf2b677` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:13.625653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] RBAC: Authorizing `identity:delete_user(user_id=055010641d1a4d0a9c57f2770bf2b677)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:13.627658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:13.684889 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '055010641d1a4d0a9c57f2770bf2b677'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:13.693439 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '055010641d1a4d0a9c57f2770bf2b677'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:13.694082 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '055010641d1a4d0a9c57f2770bf2b677'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:13.694701 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '055010641d1a4d0a9c57f2770bf2b677'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:13.710230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '055010641d1a4d0a9c57f2770bf2b677'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:13.724867 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1436/2879] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:32:13 2022] DELETE /identity/v3/users/055010641d1a4d0a9c57f2770bf2b677 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:13.732006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8586e9-a505-4bd6-8648-ffa64d04c1f7 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:13.732868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8586e9-a505-4bd6-8648-ffa64d04c1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:13.732868 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8586e9-a505-4bd6-8648-ffa64d04c1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:13.753173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-2d8586e9-a505-4bd6-8648-ffa64d04c1f7 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:13.754405 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2d8586e9-a505-4bd6-8648-ffa64d04c1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:13.823265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1444/2880] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:13.833583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:13.835084 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:13.899952 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72767bf1-e2a8-4e3c-96af-d632ef1b698e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:13.904211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7393a5a-0e32-4e87-b58d-d2ad3de1ec73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:13.904825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7393a5a-0e32-4e87-b58d-d2ad3de1ec73 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:13.905108 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7393a5a-0e32-4e87-b58d-d2ad3de1ec73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:13.905376 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7393a5a-0e32-4e87-b58d-d2ad3de1ec73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:13.916809 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7393a5a-0e32-4e87-b58d-d2ad3de1ec73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:13.918266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7393a5a-0e32-4e87-b58d-d2ad3de1ec73 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:13.929144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1437/2881] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:14.229809 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8586e9-a505-4bd6-8648-ffa64d04c1f7 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:14.232653 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-2d8586e9-a505-4bd6-8648-ffa64d04c1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:14.246107 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8586e9-a505-4bd6-8648-ffa64d04c1f7 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:14.251875 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f47ee-ef63-4093-924f-377d84337661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:14.252910 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f47ee-ef63-4093-924f-377d84337661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:14.253412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f47ee-ef63-4093-924f-377d84337661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:14.253542 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f47ee-ef63-4093-924f-377d84337661 admin admin] PATH_INFO: `/v3/projects/b14eecc2f10b455593468ccbcd52f912` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:14.254698 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f47ee-ef63-4093-924f-377d84337661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b14eecc2f10b455593468ccbcd52f912)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:14.256670 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f47ee-ef63-4093-924f-377d84337661 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:14.315297 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-227f47ee-ef63-4093-924f-377d84337661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b14eecc2f10b455593468ccbcd52f912', 'request_id': 'req-227f47ee-ef63-4093-924f-377d84337661'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:14.322494 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-227f47ee-ef63-4093-924f-377d84337661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b14eecc2f10b455593468ccbcd52f912', 'request_id': 'req-227f47ee-ef63-4093-924f-377d84337661'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:14.328360 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-227f47ee-ef63-4093-924f-377d84337661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b14eecc2f10b455593468ccbcd52f912', 'request_id': 'req-227f47ee-ef63-4093-924f-377d84337661'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:14.342904 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-227f47ee-ef63-4093-924f-377d84337661 admin admin] The token cache is being invalidate because project b14eecc2f10b455593468ccbcd52f912 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:32:14.343247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-227f47ee-ef63-4093-924f-377d84337661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:14.343884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-227f47ee-ef63-4093-924f-377d84337661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:14.345581 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1445/2882] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:32:14 2022] DELETE /identity/v3/projects/b14eecc2f10b455593468ccbcd52f912 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:14.351252 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a697744-8478-4155-842d-48c365c6a1c9 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:14.351539 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a697744-8478-4155-842d-48c365c6a1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:14.351807 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a697744-8478-4155-842d-48c365c6a1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:14.376173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-3a697744-8478-4155-842d-48c365c6a1c9 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:14.377481 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3a697744-8478-4155-842d-48c365c6a1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:14.395764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:14.443401 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1438/2883] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:14.452112 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e05e42-4920-418b-8a3a-1729c0899cae None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:14.452827 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e05e42-4920-418b-8a3a-1729c0899cae None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:14.453106 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e05e42-4920-418b-8a3a-1729c0899cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:14.475209 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-17e05e42-4920-418b-8a3a-1729c0899cae None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:14.476450 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-17e05e42-4920-418b-8a3a-1729c0899cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:14.485286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1446/2884] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:14.492780 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a697744-8478-4155-842d-48c365c6a1c9 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:14.503856 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a697744-8478-4155-842d-48c365c6a1c9 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:14.509268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8793c-f33b-4d52-8f3e-df3fa07d7262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:14.509903 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8793c-f33b-4d52-8f3e-df3fa07d7262 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:14.510184 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8793c-f33b-4d52-8f3e-df3fa07d7262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:14.510456 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8793c-f33b-4d52-8f3e-df3fa07d7262 admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:14.511133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8793c-f33b-4d52-8f3e-df3fa07d7262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:32:14.511448 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8793c-f33b-4d52-8f3e-df3fa07d7262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:14.513064 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8793c-f33b-4d52-8f3e-df3fa07d7262 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:14.525512 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1439/2885] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:32:14.529971 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e05e42-4920-418b-8a3a-1729c0899cae None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:14.540485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e05e42-4920-418b-8a3a-1729c0899cae None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:14.545173 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0a8a2-b47b-4257-b0d3-574b95f1d418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:14.545776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0a8a2-b47b-4257-b0d3-574b95f1d418 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:14.546057 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0a8a2-b47b-4257-b0d3-574b95f1d418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:14.546336 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0a8a2-b47b-4257-b0d3-574b95f1d418 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:14.547013 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0a8a2-b47b-4257-b0d3-574b95f1d418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:14.548943 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0a8a2-b47b-4257-b0d3-574b95f1d418 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:14.577661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1447/2886] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:14 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:14.582144 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8793c-f33b-4d52-8f3e-df3fa07d7262 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:14.593327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8793c-f33b-4d52-8f3e-df3fa07d7262 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:14.654826 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aad13b-b05d-4437-a8f8-11851f147b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:14.655484 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aad13b-b05d-4437-a8f8-11851f147b6e admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:14.655758 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aad13b-b05d-4437-a8f8-11851f147b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:14.656032 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aad13b-b05d-4437-a8f8-11851f147b6e admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:14.656683 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aad13b-b05d-4437-a8f8-11851f147b6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:14.658688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aad13b-b05d-4437-a8f8-11851f147b6e admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:14.687092 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1440/2887] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:14 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:14.691333 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0a8a2-b47b-4257-b0d3-574b95f1d418 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:14.702485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0a8a2-b47b-4257-b0d3-574b95f1d418 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:14.707564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6bc50-bb96-4abe-8744-7eeb29eb2f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:14.708191 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6bc50-bb96-4abe-8744-7eeb29eb2f87 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:14.708466 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6bc50-bb96-4abe-8744-7eeb29eb2f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:14.708737 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6bc50-bb96-4abe-8744-7eeb29eb2f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:14.709395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6bc50-bb96-4abe-8744-7eeb29eb2f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:14.711008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6bc50-bb96-4abe-8744-7eeb29eb2f87 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:14.725455 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1448/2888] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:14.730641 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aad13b-b05d-4437-a8f8-11851f147b6e admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:14.741994 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aad13b-b05d-4437-a8f8-11851f147b6e admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:14.747471 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4b5d2-4508-488b-a561-631098811a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:14.748097 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4b5d2-4508-488b-a561-631098811a79 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:14.748379 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4b5d2-4508-488b-a561-631098811a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:14.748812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4b5d2-4508-488b-a561-631098811a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:14.749468 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4b5d2-4508-488b-a561-631098811a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:14.751095 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4b5d2-4508-488b-a561-631098811a79 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:14.762818 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1441/2889] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:32:14.768301 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6bc50-bb96-4abe-8744-7eeb29eb2f87 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:14.781030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6bc50-bb96-4abe-8744-7eeb29eb2f87 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:14.785906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5d50-f83e-4d8e-81ae-bbceacf8d224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:14.786586 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5d50-f83e-4d8e-81ae-bbceacf8d224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:14.786862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5d50-f83e-4d8e-81ae-bbceacf8d224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:14.787186 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5d50-f83e-4d8e-81ae-bbceacf8d224 admin admin] PATH_INFO: `/v3/projects/46a7c70b6e6c4ed4bc4445900d2ab03a/users/5a28efffb7b041c5b8ee0fbbc850c483/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:14.813033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f5d50-f83e-4d8e-81ae-bbceacf8d224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a7c70b6e6c4ed4bc4445900d2ab03a, user_id=5a28efffb7b041c5b8ee0fbbc850c483, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:14.815882 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f5d50-f83e-4d8e-81ae-bbceacf8d224 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:14.831850 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1449/2890] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:14 2022] PUT /identity/v3/projects/46a7c70b6e6c4ed4bc4445900d2ab03a/users/5a28efffb7b041c5b8ee0fbbc850c483/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:14.837160 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c58bd1-f83d-4f16-a93f-73cbaa96aa77 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:14.837376 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c58bd1-f83d-4f16-a93f-73cbaa96aa77 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:14.837650 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c58bd1-f83d-4f16-a93f-73cbaa96aa77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:14.860323 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-19c58bd1-f83d-4f16-a93f-73cbaa96aa77 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:14.861597 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-19c58bd1-f83d-4f16-a93f-73cbaa96aa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:14.926387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1442/2891] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:14.937551 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5d50-f83e-4d8e-81ae-bbceacf8d224 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:14.939133 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-544f5d50-f83e-4d8e-81ae-bbceacf8d224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:14.961142 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:15.007286 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5d50-f83e-4d8e-81ae-bbceacf8d224 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:15.011171 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54097aa2-879e-4dc9-83d2-ea3881a4eec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:15.011814 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54097aa2-879e-4dc9-83d2-ea3881a4eec6 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:15.012095 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54097aa2-879e-4dc9-83d2-ea3881a4eec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:15.012386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54097aa2-879e-4dc9-83d2-ea3881a4eec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:15.022387 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54097aa2-879e-4dc9-83d2-ea3881a4eec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:15.023860 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54097aa2-879e-4dc9-83d2-ea3881a4eec6 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:15.035906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1450/2892] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:20.048141 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd04fca-183c-45f7-8a0c-37d8b451d205 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.048612 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd04fca-183c-45f7-8a0c-37d8b451d205 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.048858 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd04fca-183c-45f7-8a0c-37d8b451d205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.085004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-3cd04fca-183c-45f7-8a0c-37d8b451d205 None None] MFA Rules not processed for user `5a28efffb7b041c5b8ee0fbbc850c483`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:20.086386 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3cd04fca-183c-45f7-8a0c-37d8b451d205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:20.107013 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:20.172450 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1443/2893] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:32:20.178422 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54097aa2-879e-4dc9-83d2-ea3881a4eec6 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.191361 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54097aa2-879e-4dc9-83d2-ea3881a4eec6 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.196546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9828c01-6819-4ef0-bd01-16c8a89280f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.197365 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9828c01-6819-4ef0-bd01-16c8a89280f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.197742 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9828c01-6819-4ef0-bd01-16c8a89280f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.198121 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9828c01-6819-4ef0-bd01-16c8a89280f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.198871 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9828c01-6819-4ef0-bd01-16c8a89280f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.200955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9828c01-6819-4ef0-bd01-16c8a89280f4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.247896 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1451/2894] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:20.249513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd04fca-183c-45f7-8a0c-37d8b451d205 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.308823 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd04fca-183c-45f7-8a0c-37d8b451d205 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.382453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360523a5-f348-4898-bb42-7376d1096064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.383232 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360523a5-f348-4898-bb42-7376d1096064 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.384132 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360523a5-f348-4898-bb42-7376d1096064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.384644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360523a5-f348-4898-bb42-7376d1096064 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.385408 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360523a5-f348-4898-bb42-7376d1096064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.387920 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360523a5-f348-4898-bb42-7376d1096064 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.428732 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1444/2895] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:20.432488 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9828c01-6819-4ef0-bd01-16c8a89280f4 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.469635 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9828c01-6819-4ef0-bd01-16c8a89280f4 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.474377 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ec819-310e-44d3-93ab-c7cd794943c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.474989 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ec819-310e-44d3-93ab-c7cd794943c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.475326 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ec819-310e-44d3-93ab-c7cd794943c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.475611 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ec819-310e-44d3-93ab-c7cd794943c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.476260 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ec819-310e-44d3-93ab-c7cd794943c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.477863 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ec819-310e-44d3-93ab-c7cd794943c2 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.489897 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1452/2896] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:32:20.495016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360523a5-f348-4898-bb42-7376d1096064 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.506888 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360523a5-f348-4898-bb42-7376d1096064 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.512352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8788cc1-f416-47e6-ae7c-622c7b6d95e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.513063 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8788cc1-f416-47e6-ae7c-622c7b6d95e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.513335 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8788cc1-f416-47e6-ae7c-622c7b6d95e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.513604 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8788cc1-f416-47e6-ae7c-622c7b6d95e3 admin admin] PATH_INFO: `/v3/projects/809b895aec4449bfa7e4b6146cdc5a29/users/9f7a56da5f4e4abeb8b59362beb710b9/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.531241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8788cc1-f416-47e6-ae7c-622c7b6d95e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=809b895aec4449bfa7e4b6146cdc5a29, user_id=9f7a56da5f4e4abeb8b59362beb710b9, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.534028 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8788cc1-f416-47e6-ae7c-622c7b6d95e3 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.551017 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1445/2897] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:20 2022] PUT /identity/v3/projects/809b895aec4449bfa7e4b6146cdc5a29/users/9f7a56da5f4e4abeb8b59362beb710b9/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:20.556323 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ec819-310e-44d3-93ab-c7cd794943c2 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.568983 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ec819-310e-44d3-93ab-c7cd794943c2 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.647075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab824ea-beae-4313-9d34-f682c754fb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.647728 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab824ea-beae-4313-9d34-f682c754fb68 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.648012 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab824ea-beae-4313-9d34-f682c754fb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.648288 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab824ea-beae-4313-9d34-f682c754fb68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.648995 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab824ea-beae-4313-9d34-f682c754fb68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.650577 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab824ea-beae-4313-9d34-f682c754fb68 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.662735 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1453/2898] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:20.667730 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8788cc1-f416-47e6-ae7c-622c7b6d95e3 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.678463 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8788cc1-f416-47e6-ae7c-622c7b6d95e3 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.687020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cf283-5121-4af0-9f15-5efdf98ce3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.687020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cf283-5121-4af0-9f15-5efdf98ce3bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.687020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cf283-5121-4af0-9f15-5efdf98ce3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.687020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cf283-5121-4af0-9f15-5efdf98ce3bb admin admin] PATH_INFO: `/v3/projects/809b895aec4449bfa7e4b6146cdc5a29/users/9f7a56da5f4e4abeb8b59362beb710b9/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.691667 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3cf283-5121-4af0-9f15-5efdf98ce3bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=809b895aec4449bfa7e4b6146cdc5a29, user_id=9f7a56da5f4e4abeb8b59362beb710b9, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.694373 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3cf283-5121-4af0-9f15-5efdf98ce3bb admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.706503 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1446/2899] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:20 2022] PUT /identity/v3/projects/809b895aec4449bfa7e4b6146cdc5a29/users/9f7a56da5f4e4abeb8b59362beb710b9/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:25.641488 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9f193-95fe-427d-87d5-6e28cc19ce01 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:25.642014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9f193-95fe-427d-87d5-6e28cc19ce01 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:25.642014 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9f193-95fe-427d-87d5-6e28cc19ce01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:25.673815 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-49f9f193-95fe-427d-87d5-6e28cc19ce01 None None] MFA Rules not processed for user `9f7a56da5f4e4abeb8b59362beb710b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:25.675318 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-49f9f193-95fe-427d-87d5-6e28cc19ce01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:25.732780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:25.777595 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1454/2900] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:32:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:25.782123 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cf283-5121-4af0-9f15-5efdf98ce3bb admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:25.785239 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-2c3cf283-5121-4af0-9f15-5efdf98ce3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:25.859256 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:25.917395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cf283-5121-4af0-9f15-5efdf98ce3bb admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:25.921524 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4365738-7bbf-42b2-b3ce-8d68cdf245b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f82adce75b44c6baf85b6b96dd0728', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:25.922235 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4365738-7bbf-42b2-b3ce-8d68cdf245b3 service glance] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:25.922576 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4365738-7bbf-42b2-b3ce-8d68cdf245b3 service glance] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:25.923023 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4365738-7bbf-42b2-b3ce-8d68cdf245b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:26.000611 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4365738-7bbf-42b2-b3ce-8d68cdf245b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:26.003059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4365738-7bbf-42b2-b3ce-8d68cdf245b3 service glance] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:26.013901 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1447/2901] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:26.198782 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd75876-b037-4fa2-84c9-14ba4da68def None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:26.199094 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd75876-b037-4fa2-84c9-14ba4da68def None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:26.199430 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd75876-b037-4fa2-84c9-14ba4da68def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:26.276627 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-9bd75876-b037-4fa2-84c9-14ba4da68def None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:26.277867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-9bd75876-b037-4fa2-84c9-14ba4da68def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:26.353451 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1455/2902] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:32:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:32:26.406698 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4365738-7bbf-42b2-b3ce-8d68cdf245b3 service glance] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:26.408242 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-e4365738-7bbf-42b2-b3ce-8d68cdf245b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:26.511723 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4365738-7bbf-42b2-b3ce-8d68cdf245b3 service glance] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:26.518122 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec049d7-b857-495b-b8da-615e169239fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:26.518740 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec049d7-b857-495b-b8da-615e169239fe service swift] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:26.519019 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec049d7-b857-495b-b8da-615e169239fe service swift] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:26.519366 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec049d7-b857-495b-b8da-615e169239fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:26.531829 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec049d7-b857-495b-b8da-615e169239fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:26.533430 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec049d7-b857-495b-b8da-615e169239fe service swift] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:26.542316 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1448/2903] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:32:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:32:35.430190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457826ad-04da-42c8-8624-742a05d63d5f None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:35.430949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457826ad-04da-42c8-8624-742a05d63d5f None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:35.431687 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457826ad-04da-42c8-8624-742a05d63d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:35.466257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-457826ad-04da-42c8-8624-742a05d63d5f None None] MFA Rules not processed for user `5a28efffb7b041c5b8ee0fbbc850c483`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:35.468058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-457826ad-04da-42c8-8624-742a05d63d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:35.479709 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1456/2904] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:32:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:35.529960 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec049d7-b857-495b-b8da-615e169239fe service swift] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:35.542172 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec049d7-b857-495b-b8da-615e169239fe service swift] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:35.546574 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff665ac-871d-4286-8f7b-17a543444c50 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f82adce75b44c6baf85b6b96dd0728', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:35.547688 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff665ac-871d-4286-8f7b-17a543444c50 service glance] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:35.548118 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff665ac-871d-4286-8f7b-17a543444c50 service glance] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:35.548652 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff665ac-871d-4286-8f7b-17a543444c50 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:35.550489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-dff665ac-871d-4286-8f7b-17a543444c50 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:35.612243 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=100 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:35.642128 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff665ac-871d-4286-8f7b-17a543444c50 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:35.643664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff665ac-871d-4286-8f7b-17a543444c50 service glance] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:35.655980 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1449/2905] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:35.913891 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457826ad-04da-42c8-8624-742a05d63d5f None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:36.120257 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457826ad-04da-42c8-8624-742a05d63d5f None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:36.125278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550edeb4-967d-4ca2-b8ad-fb71a0283641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:36.126043 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550edeb4-967d-4ca2-b8ad-fb71a0283641 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:36.126898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550edeb4-967d-4ca2-b8ad-fb71a0283641 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:36.126898 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550edeb4-967d-4ca2-b8ad-fb71a0283641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:36.128653 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-550edeb4-967d-4ca2-b8ad-fb71a0283641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:36.156141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550edeb4-967d-4ca2-b8ad-fb71a0283641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:36.158799 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550edeb4-967d-4ca2-b8ad-fb71a0283641 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:36.174906 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1457/2906] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:32:36.576478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff665ac-871d-4286-8f7b-17a543444c50 service glance] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:36.588689 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff665ac-871d-4286-8f7b-17a543444c50 service glance] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:36.593320 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a21df5-271c-40e3-862d-cf52ddf81c35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:36.594003 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a21df5-271c-40e3-862d-cf52ddf81c35 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:36.594325 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a21df5-271c-40e3-862d-cf52ddf81c35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:36.594754 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a21df5-271c-40e3-862d-cf52ddf81c35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:36.596783 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-72a21df5-271c-40e3-862d-cf52ddf81c35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:36.615686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a21df5-271c-40e3-862d-cf52ddf81c35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:36.618034 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a21df5-271c-40e3-862d-cf52ddf81c35 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:36.632781 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1450/2907] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:32:38.160933 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7735454-87f5-4d16-8552-41a7f2c091fe None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:38.161508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7735454-87f5-4d16-8552-41a7f2c091fe None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:38.161508 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7735454-87f5-4d16-8552-41a7f2c091fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:38.184804 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-f7735454-87f5-4d16-8552-41a7f2c091fe None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:38.185312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-f7735454-87f5-4d16-8552-41a7f2c091fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:38.193030 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1458/2908] 10.222.0.61 () {58 vars in 1009 bytes} [Wed Sep 7 13:32:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:38.207069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a21df5-271c-40e3-862d-cf52ddf81c35 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:38.219327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a21df5-271c-40e3-862d-cf52ddf81c35 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:38.226590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a0978-9f76-40fa-9d2d-a68fbf40ae9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:38.227254 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0978-9f76-40fa-9d2d-a68fbf40ae9c service swift] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:38.227537 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0978-9f76-40fa-9d2d-a68fbf40ae9c service swift] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:38.227808 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0978-9f76-40fa-9d2d-a68fbf40ae9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:38.244640 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a0978-9f76-40fa-9d2d-a68fbf40ae9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:38.245801 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a0978-9f76-40fa-9d2d-a68fbf40ae9c service swift] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:38.256275 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1451/2909] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:32:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:32:46.552291 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7735454-87f5-4d16-8552-41a7f2c091fe None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:46.564514 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7735454-87f5-4d16-8552-41a7f2c091fe None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:46.570136 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bf658-be0d-41ee-8e73-704adeaba5ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:46.571061 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bf658-be0d-41ee-8e73-704adeaba5ca service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:46.571376 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bf658-be0d-41ee-8e73-704adeaba5ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:46.571651 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bf658-be0d-41ee-8e73-704adeaba5ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:46.573583 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1a3bf658-be0d-41ee-8e73-704adeaba5ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:46.587892 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bf658-be0d-41ee-8e73-704adeaba5ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:46.589357 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bf658-be0d-41ee-8e73-704adeaba5ca service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:46.599776 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1459/2910] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:47.380384 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a0978-9f76-40fa-9d2d-a68fbf40ae9c service swift] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:47.385487 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-eb1a0978-9f76-40fa-9d2d-a68fbf40ae9c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:47.413033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a0978-9f76-40fa-9d2d-a68fbf40ae9c service swift] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:47.418820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10e8ca-bf7a-446b-9c76-2516ae32b9ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:47.419583 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10e8ca-bf7a-446b-9c76-2516ae32b9ac service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:47.419820 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10e8ca-bf7a-446b-9c76-2516ae32b9ac service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:47.420426 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10e8ca-bf7a-446b-9c76-2516ae32b9ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:47.423774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ce10e8ca-bf7a-446b-9c76-2516ae32b9ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:47.442885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10e8ca-bf7a-446b-9c76-2516ae32b9ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:47.448552 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10e8ca-bf7a-446b-9c76-2516ae32b9ac service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:47.465545 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1452/2911] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:32:56.594703 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=97 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:57.457195 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=27 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:06.696900 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bf658-be0d-41ee-8e73-704adeaba5ca service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:06.698445 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-1a3bf658-be0d-41ee-8e73-704adeaba5ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:06.713163 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bf658-be0d-41ee-8e73-704adeaba5ca service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:06.721402 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b02d01-92c2-406f-97fb-4880c16ee6b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:06.722029 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b02d01-92c2-406f-97fb-4880c16ee6b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:06.722318 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b02d01-92c2-406f-97fb-4880c16ee6b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:06.722584 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b02d01-92c2-406f-97fb-4880c16ee6b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:06.733052 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b02d01-92c2-406f-97fb-4880c16ee6b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:06.735200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b02d01-92c2-406f-97fb-4880c16ee6b5 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:06.751045 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1460/2912] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:16.744566 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=6 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:44.854622 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10e8ca-bf7a-446b-9c76-2516ae32b9ac service nova] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:44.857177 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ce10e8ca-bf7a-446b-9c76-2516ae32b9ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:44.922433 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10e8ca-bf7a-446b-9c76-2516ae32b9ac service nova] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:44.929769 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901f5a5-b1ca-4a88-b208-22eccda515b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:44.930814 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901f5a5-b1ca-4a88-b208-22eccda515b5 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:44.931089 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901f5a5-b1ca-4a88-b208-22eccda515b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:44.931592 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901f5a5-b1ca-4a88-b208-22eccda515b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:44.933729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-d901f5a5-b1ca-4a88-b208-22eccda515b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:44.948649 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901f5a5-b1ca-4a88-b208-22eccda515b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:44.950352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901f5a5-b1ca-4a88-b208-22eccda515b5 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:44.965173 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1453/2913] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:52.551429 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec808bbd-c828-45a3-9edb-8dc0f2024d62 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:52.552011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec808bbd-c828-45a3-9edb-8dc0f2024d62 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:52.552175 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec808bbd-c828-45a3-9edb-8dc0f2024d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:52.585517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-ec808bbd-c828-45a3-9edb-8dc0f2024d62 None None] MFA Rules not processed for user `d71863f18330414f9b6238b7e8903701`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:52.586736 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ec808bbd-c828-45a3-9edb-8dc0f2024d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:52.604636 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1461/2914] 10.222.0.61 () {58 vars in 967 bytes} [Wed Sep 7 13:34:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:52.654197 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901f5a5-b1ca-4a88-b208-22eccda515b5 service nova] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:52.672268 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901f5a5-b1ca-4a88-b208-22eccda515b5 service nova] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:52.682905 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc0dac-9c0c-4efc-9b7e-55bed05167ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f886e0999d34bf9870b448772cc38a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:52.683555 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc0dac-9c0c-4efc-9b7e-55bed05167ed service swift] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:52.683832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc0dac-9c0c-4efc-9b7e-55bed05167ed service swift] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:52.684111 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc0dac-9c0c-4efc-9b7e-55bed05167ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:52.692983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc0dac-9c0c-4efc-9b7e-55bed05167ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:52.695360 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc0dac-9c0c-4efc-9b7e-55bed05167ed service swift] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:52.706370 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1454/2915] 10.222.0.61 () {60 vars in 1384 bytes} [Wed Sep 7 13:34:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:34:56.549561 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec808bbd-c828-45a3-9edb-8dc0f2024d62 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:56.551848 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ec808bbd-c828-45a3-9edb-8dc0f2024d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:56.650378 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec808bbd-c828-45a3-9edb-8dc0f2024d62 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:56.658829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ab4fd-a533-4e51-b405-2eb4d1282da5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:56.659490 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ab4fd-a533-4e51-b405-2eb4d1282da5 service placement] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:56.659771 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ab4fd-a533-4e51-b405-2eb4d1282da5 service placement] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:56.660044 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ab4fd-a533-4e51-b405-2eb4d1282da5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:56.661830 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b06ab4fd-a533-4e51-b405-2eb4d1282da5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:56.676263 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ab4fd-a533-4e51-b405-2eb4d1282da5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:56.678234 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ab4fd-a533-4e51-b405-2eb4d1282da5 service placement] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:56.691195 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1462/2916] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:59.783771 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc0dac-9c0c-4efc-9b7e-55bed05167ed service swift] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:59.894327 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc0dac-9c0c-4efc-9b7e-55bed05167ed service swift] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:59.899589 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:59.900223 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:59.900506 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:59.900774 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] PATH_INFO: `/v3/users/5a28efffb7b041c5b8ee0fbbc850c483` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:59.901983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a28efffb7b041c5b8ee0fbbc850c483)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:59.903959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:59.959455 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a28efffb7b041c5b8ee0fbbc850c483'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:59.968058 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a28efffb7b041c5b8ee0fbbc850c483'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:59.968805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a28efffb7b041c5b8ee0fbbc850c483'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:59.969596 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a28efffb7b041c5b8ee0fbbc850c483'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:59.985043 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a28efffb7b041c5b8ee0fbbc850c483'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:59.999224 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1455/2917] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:59 2022] DELETE /identity/v3/users/5a28efffb7b041c5b8ee0fbbc850c483 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:00.187260 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ab4fd-a533-4e51-b405-2eb4d1282da5 service placement] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:00.204507 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ab4fd-a533-4e51-b405-2eb4d1282da5 service placement] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:00.273312 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:00.274058 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:00.274340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:00.274616 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] PATH_INFO: `/v3/projects/46a7c70b6e6c4ed4bc4445900d2ab03a` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:00.275825 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=46a7c70b6e6c4ed4bc4445900d2ab03a)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:00.277770 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:00.328657 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46a7c70b6e6c4ed4bc4445900d2ab03a', 'request_id': 'req-556423e6-da69-4ce5-a927-83aa00dda8fb'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.335735 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46a7c70b6e6c4ed4bc4445900d2ab03a', 'request_id': 'req-556423e6-da69-4ce5-a927-83aa00dda8fb'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.342262 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46a7c70b6e6c4ed4bc4445900d2ab03a', 'request_id': 'req-556423e6-da69-4ce5-a927-83aa00dda8fb'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.355269 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:00.369033 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] The token cache is being invalidate because project 46a7c70b6e6c4ed4bc4445900d2ab03a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:35:00.369415 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.371631 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.373327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1463/2918] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:35:00 2022] DELETE /identity/v3/projects/46a7c70b6e6c4ed4bc4445900d2ab03a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:00.377348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:00.378750 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:00.393731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3a550-96d2-4795-b80f-0f8b002a418d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:00.398918 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:00.458059 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf475b-0c94-4b71-b5e9-d553fa45fa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:00.458620 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf475b-0c94-4b71-b5e9-d553fa45fa13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:00.458897 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf475b-0c94-4b71-b5e9-d553fa45fa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:00.459211 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf475b-0c94-4b71-b5e9-d553fa45fa13 admin admin] PATH_INFO: `/v3/users/9f7a56da5f4e4abeb8b59362beb710b9` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:00.460411 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf475b-0c94-4b71-b5e9-d553fa45fa13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f7a56da5f4e4abeb8b59362beb710b9)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:00.462352 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf475b-0c94-4b71-b5e9-d553fa45fa13 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:00.511967 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5ccf475b-0c94-4b71-b5e9-d553fa45fa13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f7a56da5f4e4abeb8b59362beb710b9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.519434 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5ccf475b-0c94-4b71-b5e9-d553fa45fa13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f7a56da5f4e4abeb8b59362beb710b9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.520066 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5ccf475b-0c94-4b71-b5e9-d553fa45fa13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f7a56da5f4e4abeb8b59362beb710b9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.520674 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5ccf475b-0c94-4b71-b5e9-d553fa45fa13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f7a56da5f4e4abeb8b59362beb710b9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.532695 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-5ccf475b-0c94-4b71-b5e9-d553fa45fa13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f7a56da5f4e4abeb8b59362beb710b9'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.545020 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1456/2919] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:35:00 2022] DELETE /identity/v3/users/9f7a56da5f4e4abeb8b59362beb710b9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:00.723642 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:00.725964 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:00.744939 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556423e6-da69-4ce5-a927-83aa00dda8fb admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:00.806123 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4429a-68cc-4e56-9a49-298117f74800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:00.806762 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4429a-68cc-4e56-9a49-298117f74800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:00.807172 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4429a-68cc-4e56-9a49-298117f74800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:00.807481 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4429a-68cc-4e56-9a49-298117f74800 admin admin] PATH_INFO: `/v3/projects/809b895aec4449bfa7e4b6146cdc5a29` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:00.808688 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4429a-68cc-4e56-9a49-298117f74800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=809b895aec4449bfa7e4b6146cdc5a29)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:00.810620 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4429a-68cc-4e56-9a49-298117f74800 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:00.857972 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-86b4429a-68cc-4e56-9a49-298117f74800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '809b895aec4449bfa7e4b6146cdc5a29', 'request_id': 'req-86b4429a-68cc-4e56-9a49-298117f74800'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.864373 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-86b4429a-68cc-4e56-9a49-298117f74800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '809b895aec4449bfa7e4b6146cdc5a29', 'request_id': 'req-86b4429a-68cc-4e56-9a49-298117f74800'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.870726 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-86b4429a-68cc-4e56-9a49-298117f74800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '809b895aec4449bfa7e4b6146cdc5a29', 'request_id': 'req-86b4429a-68cc-4e56-9a49-298117f74800'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.883822 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-86b4429a-68cc-4e56-9a49-298117f74800 admin admin] The token cache is being invalidate because project 809b895aec4449bfa7e4b6146cdc5a29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:35:00.884205 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-86b4429a-68cc-4e56-9a49-298117f74800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.884825 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-86b4429a-68cc-4e56-9a49-298117f74800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:00.886376 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1464/2920] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:35:00 2022] DELETE /identity/v3/projects/809b895aec4449bfa7e4b6146cdc5a29 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:35:00.891435 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f486691-bfe8-4e10-9f6d-a696b41f7e5b None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:00.892154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f486691-bfe8-4e10-9f6d-a696b41f7e5b None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:00.892154 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f486691-bfe8-4e10-9f6d-a696b41f7e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:00.956226 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-1f486691-bfe8-4e10-9f6d-a696b41f7e5b None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:00.957525 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1f486691-bfe8-4e10-9f6d-a696b41f7e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:01.010885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:01.021310 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1457/2921] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:35:01.029347 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e6d8f-1c54-4c24-b46d-daac8b006602 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:01.029626 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e6d8f-1c54-4c24-b46d-daac8b006602 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:01.029901 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e6d8f-1c54-4c24-b46d-daac8b006602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:01.050843 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-ad5e6d8f-1c54-4c24-b46d-daac8b006602 None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:01.052278 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-ad5e6d8f-1c54-4c24-b46d-daac8b006602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:01.062166 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1465/2922] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:01.067000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f486691-bfe8-4e10-9f6d-a696b41f7e5b None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:01.078000 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f486691-bfe8-4e10-9f6d-a696b41f7e5b None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:01.082836 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed5b18-061c-4ed8-a576-0e32363d1d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:01.083489 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed5b18-061c-4ed8-a576-0e32363d1d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:01.083821 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed5b18-061c-4ed8-a576-0e32363d1d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:01.084093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed5b18-061c-4ed8-a576-0e32363d1d7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:01.084731 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed5b18-061c-4ed8-a576-0e32363d1d7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81795) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:35:01.085004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed5b18-061c-4ed8-a576-0e32363d1d7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:01.087227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed5b18-061c-4ed8-a576-0e32363d1d7a admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:01.099165 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1458/2923] 10.222.0.61 () {58 vars in 1153 bytes} [Wed Sep 7 13:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:35:01.103697 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e6d8f-1c54-4c24-b46d-daac8b006602 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:01.113345 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e6d8f-1c54-4c24-b46d-daac8b006602 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:01.119238 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcbfaa-6f35-4b00-bf1d-1a96e6a9ba12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:01.119845 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcbfaa-6f35-4b00-bf1d-1a96e6a9ba12 admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:01.120120 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcbfaa-6f35-4b00-bf1d-1a96e6a9ba12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:01.120385 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcbfaa-6f35-4b00-bf1d-1a96e6a9ba12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:01.121016 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcbfaa-6f35-4b00-bf1d-1a96e6a9ba12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:01.122890 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcbfaa-6f35-4b00-bf1d-1a96e6a9ba12 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:01.160599 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1466/2924] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:35:01 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:35:01.164478 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed5b18-061c-4ed8-a576-0e32363d1d7a admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:01.175825 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed5b18-061c-4ed8-a576-0e32363d1d7a admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:01.236664 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d99e4-a07f-4155-bcae-366a80c62252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:01.237283 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d99e4-a07f-4155-bcae-366a80c62252 admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:01.237564 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d99e4-a07f-4155-bcae-366a80c62252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:01.237835 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d99e4-a07f-4155-bcae-366a80c62252 admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:01.238486 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623d99e4-a07f-4155-bcae-366a80c62252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:01.240458 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623d99e4-a07f-4155-bcae-366a80c62252 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:01.272388 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1459/2925] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:35:01 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:35:01.276565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcbfaa-6f35-4b00-bf1d-1a96e6a9ba12 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:01.287826 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcbfaa-6f35-4b00-bf1d-1a96e6a9ba12 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:01.292548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972e8da-6711-4d03-8083-6aa683a069a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:01.293157 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972e8da-6711-4d03-8083-6aa683a069a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:01.293432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972e8da-6711-4d03-8083-6aa683a069a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:01.293699 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972e8da-6711-4d03-8083-6aa683a069a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:01.294342 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972e8da-6711-4d03-8083-6aa683a069a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:01.295984 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972e8da-6711-4d03-8083-6aa683a069a4 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:01.307431 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1467/2926] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:35:01.312116 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d99e4-a07f-4155-bcae-366a80c62252 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:01.324113 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d99e4-a07f-4155-bcae-366a80c62252 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:01.329378 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3132e-bd01-404f-9841-93fd39c12e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:01.330014 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3132e-bd01-404f-9841-93fd39c12e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:01.330297 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3132e-bd01-404f-9841-93fd39c12e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:01.330569 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3132e-bd01-404f-9841-93fd39c12e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:01.331308 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3132e-bd01-404f-9841-93fd39c12e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:01.333040 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3132e-bd01-404f-9841-93fd39c12e97 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:01.346339 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1460/2927] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:35:01.355446 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972e8da-6711-4d03-8083-6aa683a069a4 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:01.366693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972e8da-6711-4d03-8083-6aa683a069a4 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:01.371737 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8798eb-1908-477f-a0c3-3daa851a7a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:01.372412 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8798eb-1908-477f-a0c3-3daa851a7a44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:01.372689 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8798eb-1908-477f-a0c3-3daa851a7a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:01.372959 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8798eb-1908-477f-a0c3-3daa851a7a44 admin admin] PATH_INFO: `/v3/projects/e4fd5ac76e47431eb9ed60b34784d2e3/users/923df66d00e04d73aab0a82bc671cb13/roles/86f185569d134d80841142ae8e9d3f47` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:01.394064 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8798eb-1908-477f-a0c3-3daa851a7a44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fd5ac76e47431eb9ed60b34784d2e3, user_id=923df66d00e04d73aab0a82bc671cb13, role_id=86f185569d134d80841142ae8e9d3f47)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:01.396787 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8798eb-1908-477f-a0c3-3daa851a7a44 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:01.412406 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1468/2928] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:35:01 2022] PUT /identity/v3/projects/e4fd5ac76e47431eb9ed60b34784d2e3/users/923df66d00e04d73aab0a82bc671cb13/roles/86f185569d134d80841142ae8e9d3f47 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:01.418189 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e110abd-335e-4a17-a498-43ee774d7fac None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:01.418459 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e110abd-335e-4a17-a498-43ee774d7fac None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:01.418726 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e110abd-335e-4a17-a498-43ee774d7fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:01.439322 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-1e110abd-335e-4a17-a498-43ee774d7fac None None] MFA Rules not processed for user `8bfb2d4384094e9e8a8b910061e5c9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:01.440543 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1e110abd-335e-4a17-a498-43ee774d7fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:01.507565 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1461/2929] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:01.517484 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8798eb-1908-477f-a0c3-3daa851a7a44 admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:01.527287 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-fa8798eb-1908-477f-a0c3-3daa851a7a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:01.535602 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:01.585839 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8798eb-1908-477f-a0c3-3daa851a7a44 admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:01.589522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f2d22-f008-45ed-bbeb-6575d3f0fa34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:01.590211 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2d22-f008-45ed-bbeb-6575d3f0fa34 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:01.590483 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2d22-f008-45ed-bbeb-6575d3f0fa34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:01.590661 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2d22-f008-45ed-bbeb-6575d3f0fa34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:01.600711 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f2d22-f008-45ed-bbeb-6575d3f0fa34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:01.602564 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f2d22-f008-45ed-bbeb-6575d3f0fa34 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:01.612613 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1469/2930] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:35:06.377644 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121102e-ca21-4d97-b8f1-b673dcd27bf3 None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:06.378260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121102e-ca21-4d97-b8f1-b673dcd27bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:06.378260 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121102e-ca21-4d97-b8f1-b673dcd27bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:06.416343 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-8121102e-ca21-4d97-b8f1-b673dcd27bf3 None None] MFA Rules not processed for user `923df66d00e04d73aab0a82bc671cb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:06.417561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8121102e-ca21-4d97-b8f1-b673dcd27bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:06.540357 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:06.560523 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1462/2931] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:35:06.566075 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f2d22-f008-45ed-bbeb-6575d3f0fa34 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:06.579504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f2d22-f008-45ed-bbeb-6575d3f0fa34 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:06.586565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c387c2-742d-46b7-8051-47091d84236e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:06.586565 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c387c2-742d-46b7-8051-47091d84236e admin admin] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:06.587031 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c387c2-742d-46b7-8051-47091d84236e admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:06.587472 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c387c2-742d-46b7-8051-47091d84236e admin admin] PATH_INFO: `/v3/projects` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:06.588008 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c387c2-742d-46b7-8051-47091d84236e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:06.590054 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c387c2-742d-46b7-8051-47091d84236e admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:06.634513 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1470/2932] 10.222.0.61 () {60 vars in 1153 bytes} [Wed Sep 7 13:35:06 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:35:06.639492 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8121102e-ca21-4d97-b8f1-b673dcd27bf3 None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:06.653713 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8121102e-ca21-4d97-b8f1-b673dcd27bf3 None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:06.746033 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680db58e-bb44-48e7-8dea-875647d170fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:06.746787 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680db58e-bb44-48e7-8dea-875647d170fb admin admin] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:06.746992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680db58e-bb44-48e7-8dea-875647d170fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:06.747299 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680db58e-bb44-48e7-8dea-875647d170fb admin admin] PATH_INFO: `/v3/users` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:06.747958 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680db58e-bb44-48e7-8dea-875647d170fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:06.749945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680db58e-bb44-48e7-8dea-875647d170fb admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:06.779968 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1463/2933] 10.222.0.61 () {60 vars in 1144 bytes} [Wed Sep 7 13:35:06 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:35:06.784522 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c387c2-742d-46b7-8051-47091d84236e admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:06.804838 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c387c2-742d-46b7-8051-47091d84236e admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:06.812247 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f459afb-cbd6-4336-9ee1-fb4f4a96fcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:06.812905 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f459afb-cbd6-4336-9ee1-fb4f4a96fcec admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:06.813184 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f459afb-cbd6-4336-9ee1-fb4f4a96fcec admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:06.813553 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f459afb-cbd6-4336-9ee1-fb4f4a96fcec admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:06.814287 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f459afb-cbd6-4336-9ee1-fb4f4a96fcec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:06.815925 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f459afb-cbd6-4336-9ee1-fb4f4a96fcec admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:06.827945 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1471/2934] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:35:06.833663 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680db58e-bb44-48e7-8dea-875647d170fb admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:06.849628 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680db58e-bb44-48e7-8dea-875647d170fb admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:06.857320 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a9647-7f7c-4af2-b40d-de2eb1008c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:06.858231 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a9647-7f7c-4af2-b40d-de2eb1008c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:06.858529 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a9647-7f7c-4af2-b40d-de2eb1008c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:06.859264 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a9647-7f7c-4af2-b40d-de2eb1008c1d admin admin] PATH_INFO: `/v3/projects/13833edf7e68484390c47d0f7c797bad/users/63972a1995e843d4a077a19ef92146bc/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:06.878422 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a9647-7f7c-4af2-b40d-de2eb1008c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=13833edf7e68484390c47d0f7c797bad, user_id=63972a1995e843d4a077a19ef92146bc, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:06.881658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a9647-7f7c-4af2-b40d-de2eb1008c1d admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:06.897561 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1464/2935] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:35:06 2022] PUT /identity/v3/projects/13833edf7e68484390c47d0f7c797bad/users/63972a1995e843d4a077a19ef92146bc/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:35:06.901801 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f459afb-cbd6-4336-9ee1-fb4f4a96fcec admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:06.915644 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f459afb-cbd6-4336-9ee1-fb4f4a96fcec admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:06.991358 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c932c79-a849-4c1b-8a7d-c58344bcd3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:06.991989 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c932c79-a849-4c1b-8a7d-c58344bcd3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:06.992265 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c932c79-a849-4c1b-8a7d-c58344bcd3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:06.992532 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c932c79-a849-4c1b-8a7d-c58344bcd3b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:06.993182 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c932c79-a849-4c1b-8a7d-c58344bcd3b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:06.994780 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c932c79-a849-4c1b-8a7d-c58344bcd3b9 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:07.005862 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1472/2936] 10.222.0.61 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:35:07.012557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a9647-7f7c-4af2-b40d-de2eb1008c1d admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:07.023756 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a9647-7f7c-4af2-b40d-de2eb1008c1d admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:07.028976 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779ad63-9f09-4cb4-9589-399b82938d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:07.030507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779ad63-9f09-4cb4-9589-399b82938d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:07.030507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779ad63-9f09-4cb4-9589-399b82938d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:07.030507 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779ad63-9f09-4cb4-9589-399b82938d2f admin admin] PATH_INFO: `/v3/projects/13833edf7e68484390c47d0f7c797bad/users/63972a1995e843d4a077a19ef92146bc/roles/40c2d707402d45a0b996aecfd1f11a2f` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:07.032990 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779ad63-9f09-4cb4-9589-399b82938d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=13833edf7e68484390c47d0f7c797bad, user_id=63972a1995e843d4a077a19ef92146bc, role_id=40c2d707402d45a0b996aecfd1f11a2f)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:07.035985 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779ad63-9f09-4cb4-9589-399b82938d2f admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:07.051446 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1465/2937] 10.222.0.61 () {60 vars in 1483 bytes} [Wed Sep 7 13:35:07 2022] PUT /identity/v3/projects/13833edf7e68484390c47d0f7c797bad/users/63972a1995e843d4a077a19ef92146bc/roles/40c2d707402d45a0b996aecfd1f11a2f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:13.168126 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4971be1-1b88-4343-a5ad-a526ed0071b0 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:13.169392 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4971be1-1b88-4343-a5ad-a526ed0071b0 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:13.169741 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4971be1-1b88-4343-a5ad-a526ed0071b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:13.217226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-b4971be1-1b88-4343-a5ad-a526ed0071b0 None None] MFA Rules not processed for user `63972a1995e843d4a077a19ef92146bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:13.219548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-b4971be1-1b88-4343-a5ad-a526ed0071b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:13.333474 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1473/2938] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:35:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:13.356841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779ad63-9f09-4cb4-9589-399b82938d2f admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:13.360743 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-3779ad63-9f09-4cb4-9589-399b82938d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:13.487528 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779ad63-9f09-4cb4-9589-399b82938d2f admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:13.494841 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d73675-3265-41a7-a08d-66a12093a837 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:13.495522 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d73675-3265-41a7-a08d-66a12093a837 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:13.495980 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d73675-3265-41a7-a08d-66a12093a837 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:13.496348 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d73675-3265-41a7-a08d-66a12093a837 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:13.553617 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:13.576653 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d73675-3265-41a7-a08d-66a12093a837 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:13.578288 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d73675-3265-41a7-a08d-66a12093a837 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:13.590161 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1466/2939] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:35:14.811382 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b2a2e-377d-428f-8463-433ca7437a08 None None] REQUEST_METHOD: `POST` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:14.811955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b2a2e-377d-428f-8463-433ca7437a08 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:14.811955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b2a2e-377d-428f-8463-433ca7437a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:14.820861 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:14.838859 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.auth.core [None req-063b2a2e-377d-428f-8463-433ca7437a08 None None] MFA Rules not processed for user `923df66d00e04d73aab0a82bc671cb13`. Rule list: `[]` (Enabled: `True`). {{(pid=81796) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:14.840055 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-063b2a2e-377d-428f-8463-433ca7437a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:14.848454 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1474/2940] 10.222.0.61 () {58 vars in 958 bytes} [Wed Sep 7 13:35:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:23.585474 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=8 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:24.836515 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=2 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:29.208479 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d73675-3265-41a7-a08d-66a12093a837 service nova] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:29.210133 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-42d73675-3265-41a7-a08d-66a12093a837 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:29.300461 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d73675-3265-41a7-a08d-66a12093a837 service nova] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:29.305832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e856b-449c-407e-bb5b-497bcaad6e23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:29.306488 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e856b-449c-407e-bb5b-497bcaad6e23 service placement] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:29.306832 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e856b-449c-407e-bb5b-497bcaad6e23 service placement] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:29.307191 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e856b-449c-407e-bb5b-497bcaad6e23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:29.309685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-ca5e856b-449c-407e-bb5b-497bcaad6e23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:29.321615 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e856b-449c-407e-bb5b-497bcaad6e23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:29.323232 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e856b-449c-407e-bb5b-497bcaad6e23 service placement] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:29.334152 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1467/2941] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:35:31.657488 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b2a2e-377d-428f-8463-433ca7437a08 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:31.719638 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b2a2e-377d-428f-8463-433ca7437a08 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:31.723232 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056239cd-0ce7-4e46-bd8d-30d54e6e7cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:31.723830 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056239cd-0ce7-4e46-bd8d-30d54e6e7cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:31.724109 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056239cd-0ce7-4e46-bd8d-30d54e6e7cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:31.724622 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056239cd-0ce7-4e46-bd8d-30d54e6e7cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:31.726397 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-056239cd-0ce7-4e46-bd8d-30d54e6e7cd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:31.736517 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056239cd-0ce7-4e46-bd8d-30d54e6e7cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:31.737955 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056239cd-0ce7-4e46-bd8d-30d54e6e7cd3 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:31.747241 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1475/2942] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:35:33.217868 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e856b-449c-407e-bb5b-497bcaad6e23 service placement] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:33.233208 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e856b-449c-407e-bb5b-497bcaad6e23 service placement] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:33.238784 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550c815-b098-4ad5-a1c0-e08974276174 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:33.239453 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550c815-b098-4ad5-a1c0-e08974276174 service placement] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:33.239800 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550c815-b098-4ad5-a1c0-e08974276174 service placement] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:33.240025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550c815-b098-4ad5-a1c0-e08974276174 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:33.241885 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-8550c815-b098-4ad5-a1c0-e08974276174 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:33.256070 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550c815-b098-4ad5-a1c0-e08974276174 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:33.259016 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550c815-b098-4ad5-a1c0-e08974276174 service placement] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:33.269992 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1468/2943] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:35:41.742846 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=26 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:43.264541 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=47 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:46.749386 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056239cd-0ce7-4e46-bd8d-30d54e6e7cd3 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:46.753224 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-056239cd-0ce7-4e46-bd8d-30d54e6e7cd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:46.771122 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056239cd-0ce7-4e46-bd8d-30d54e6e7cd3 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:46.777902 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752790e4-77c7-4603-b9ee-2be725b89537 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:46.778521 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752790e4-77c7-4603-b9ee-2be725b89537 service nova] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:46.778805 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752790e4-77c7-4603-b9ee-2be725b89537 service nova] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:46.779082 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752790e4-77c7-4603-b9ee-2be725b89537 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:46.780879 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-752790e4-77c7-4603-b9ee-2be725b89537 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:46.799198 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752790e4-77c7-4603-b9ee-2be725b89537 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:46.801133 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752790e4-77c7-4603-b9ee-2be725b89537 service nova] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:46.812717 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1476/2944] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:35:50.385729 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550c815-b098-4ad5-a1c0-e08974276174 service placement] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:50.395854 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550c815-b098-4ad5-a1c0-e08974276174 service placement] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:50.400025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706f42f-f30a-4142-a027-d3c597309f39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:50.400686 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706f42f-f30a-4142-a027-d3c597309f39 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:50.401004 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706f42f-f30a-4142-a027-d3c597309f39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:50.401346 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706f42f-f30a-4142-a027-d3c597309f39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:50.411391 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706f42f-f30a-4142-a027-d3c597309f39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:50.412803 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706f42f-f30a-4142-a027-d3c597309f39 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:50.422370 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1469/2945] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:35:56.807548 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=6 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:36:00.418042 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=6 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:51.289740 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752790e4-77c7-4603-b9ee-2be725b89537 service nova] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:51.300504 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752790e4-77c7-4603-b9ee-2be725b89537 service nova] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:51.304698 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59107bb4-c25b-4bd0-96bd-8093f76f54b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:51.305311 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59107bb4-c25b-4bd0-96bd-8093f76f54b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:51.305570 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59107bb4-c25b-4bd0-96bd-8093f76f54b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:51.305832 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59107bb4-c25b-4bd0-96bd-8093f76f54b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:51.307615 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-59107bb4-c25b-4bd0-96bd-8093f76f54b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:51.325304 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59107bb4-c25b-4bd0-96bd-8093f76f54b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:51.326667 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59107bb4-c25b-4bd0-96bd-8093f76f54b0 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:51.339273 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1477/2946] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:37:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:37:52.553629 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706f42f-f30a-4142-a027-d3c597309f39 service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:52.567837 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706f42f-f30a-4142-a027-d3c597309f39 service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:52.571812 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb40060-a4b7-45ba-9dcb-aa016c4197df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:52.572680 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb40060-a4b7-45ba-9dcb-aa016c4197df service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:52.572959 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb40060-a4b7-45ba-9dcb-aa016c4197df service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:52.573230 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb40060-a4b7-45ba-9dcb-aa016c4197df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:52.575714 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-efb40060-a4b7-45ba-9dcb-aa016c4197df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:52.591684 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb40060-a4b7-45ba-9dcb-aa016c4197df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:52.593134 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb40060-a4b7-45ba-9dcb-aa016c4197df service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:52.606651 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1470/2947] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:37:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:38:01.333340 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=6 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:38:01.597635 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59107bb4-c25b-4bd0-96bd-8093f76f54b0 service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:01.606725 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59107bb4-c25b-4bd0-96bd-8093f76f54b0 service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:01.613271 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41308bfd-25c5-499f-b6b6-ea5431af9538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:01.613884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41308bfd-25c5-499f-b6b6-ea5431af9538 service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:01.614165 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41308bfd-25c5-499f-b6b6-ea5431af9538 service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:01.614433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41308bfd-25c5-499f-b6b6-ea5431af9538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:01.616295 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-41308bfd-25c5-499f-b6b6-ea5431af9538 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:01.626786 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41308bfd-25c5-499f-b6b6-ea5431af9538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:01.628256 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41308bfd-25c5-499f-b6b6-ea5431af9538 service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:01.637322 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1478/2948] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:38:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:38:02.599971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=6 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:38:06.088614 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb40060-a4b7-45ba-9dcb-aa016c4197df service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:06.091150 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-efb40060-a4b7-45ba-9dcb-aa016c4197df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:06.104550 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb40060-a4b7-45ba-9dcb-aa016c4197df service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:06.109062 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa993a66-70c6-485e-94a0-090216fac390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:06.109692 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa993a66-70c6-485e-94a0-090216fac390 service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:06.110094 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa993a66-70c6-485e-94a0-090216fac390 service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:06.110241 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa993a66-70c6-485e-94a0-090216fac390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:06.112075 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-fa993a66-70c6-485e-94a0-090216fac390 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:06.122276 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa993a66-70c6-485e-94a0-090216fac390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:06.123969 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa993a66-70c6-485e-94a0-090216fac390 service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:06.132782 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1471/2949] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:38:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:38:07.244479 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386448df-6d39-49f2-9aa1-09f6971a0543 None None] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:07.245200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386448df-6d39-49f2-9aa1-09f6971a0543 None None] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:07.245200 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386448df-6d39-49f2-9aa1-09f6971a0543 None None] PATH_INFO: `/` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:07.246434 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1479/2950] 10.222.0.61 () {54 vars in 921 bytes} [Wed Sep 7 13:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:38:07.252035 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81d9c1-345b-4b66-974a-13a30d12dcda None None] REQUEST_METHOD: `POST` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:07.252582 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81d9c1-345b-4b66-974a-13a30d12dcda None None] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:07.252692 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81d9c1-345b-4b66-974a-13a30d12dcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:07.294454 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.auth.core [None req-7a81d9c1-345b-4b66-974a-13a30d12dcda None None] MFA Rules not processed for user `ba93e2d4aaed4c5595d5fd7363626bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81795) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:38:07.295728 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-7a81d9c1-345b-4b66-974a-13a30d12dcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:07.305330 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1472/2951] 10.222.0.61 () {58 vars in 1018 bytes} [Wed Sep 7 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:38:07.316400 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386448df-6d39-49f2-9aa1-09f6971a0543 None None] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:07.328053 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386448df-6d39-49f2-9aa1-09f6971a0543 None None] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:07.331874 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b141a-96b1-4268-86c9-f0e36f3c0a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:07.332476 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b141a-96b1-4268-86c9-f0e36f3c0a5f service neutron] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:07.332756 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b141a-96b1-4268-86c9-f0e36f3c0a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:07.333028 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b141a-96b1-4268-86c9-f0e36f3c0a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:07.345038 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b141a-96b1-4268-86c9-f0e36f3c0a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:07.346480 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b141a-96b1-4268-86c9-f0e36f3c0a5f service neutron] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:07.359864 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1480/2952] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:38:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:38:17.291804 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=15 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:38:17.354852 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=12 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:50.547513 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81d9c1-345b-4b66-974a-13a30d12dcda None None] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:50.560199 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81d9c1-345b-4b66-974a-13a30d12dcda None None] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:50.567250 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753c79d-2703-4712-a03d-aaea21656bb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba93e2d4aaed4c5595d5fd7363626bcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:50.567988 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753c79d-2703-4712-a03d-aaea21656bb7 service nova] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:50.568341 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753c79d-2703-4712-a03d-aaea21656bb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:50.568685 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753c79d-2703-4712-a03d-aaea21656bb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:50.570569 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-b753c79d-2703-4712-a03d-aaea21656bb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:50.585630 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753c79d-2703-4712-a03d-aaea21656bb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:50.588557 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753c79d-2703-4712-a03d-aaea21656bb7 service nova] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:50.603093 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1473/2953] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:39:58.477952 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b141a-96b1-4268-86c9-f0e36f3c0a5f service neutron] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:58.480598 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-fd7b141a-96b1-4268-86c9-f0e36f3c0a5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:58.499197 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b141a-96b1-4268-86c9-f0e36f3c0a5f service neutron] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:58.506433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61a2ca-8c88-49ee-842d-31511bc2c7d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:58.507534 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61a2ca-8c88-49ee-842d-31511bc2c7d6 service placement] REQUEST_METHOD: `GET` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:58.508272 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61a2ca-8c88-49ee-842d-31511bc2c7d6 service placement] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:58.508812 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61a2ca-8c88-49ee-842d-31511bc2c7d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:58.511321 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-dc61a2ca-8c88-49ee-842d-31511bc2c7d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:58.527424 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61a2ca-8c88-49ee-842d-31511bc2c7d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:58.532103 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61a2ca-8c88-49ee-842d-31511bc2c7d6 service placement] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:58.575888 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1481/2954] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:00.596427 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=6 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:01.472313 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753c79d-2703-4712-a03d-aaea21656bb7 service nova] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:01.486227 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753c79d-2703-4712-a03d-aaea21656bb7 service nova] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:01.491945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97d082-401d-40ba-84ce-8e5cd0603151 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:01.493266 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97d082-401d-40ba-84ce-8e5cd0603151 service placement] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:01.493609 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97d082-401d-40ba-84ce-8e5cd0603151 service placement] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:01.493971 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97d082-401d-40ba-84ce-8e5cd0603151 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:01.495861 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-dd97d082-401d-40ba-84ce-8e5cd0603151 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:01.509803 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97d082-401d-40ba-84ce-8e5cd0603151 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:01.512218 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97d082-401d-40ba-84ce-8e5cd0603151 service placement] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:01.524441 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1474/2955] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:01.537141 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61a2ca-8c88-49ee-842d-31511bc2c7d6 service placement] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:01.606691 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61a2ca-8c88-49ee-842d-31511bc2c7d6 service placement] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:01.611226 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:01.611867 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:01.612154 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:01.612432 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] PATH_INFO: `/v3/users/923df66d00e04d73aab0a82bc671cb13` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:01.613556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] RBAC: Authorizing `identity:delete_user(user_id=923df66d00e04d73aab0a82bc671cb13)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:01.615523 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:01.662905 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '923df66d00e04d73aab0a82bc671cb13'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:01.675932 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '923df66d00e04d73aab0a82bc671cb13'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:01.683884 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '923df66d00e04d73aab0a82bc671cb13'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:01.684781 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '923df66d00e04d73aab0a82bc671cb13'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:01.685556 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '923df66d00e04d73aab0a82bc671cb13'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:01.698866 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1482/2956] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:40:01 2022] DELETE /identity/v3/users/923df66d00e04d73aab0a82bc671cb13 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:01.708658 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97d082-401d-40ba-84ce-8e5cd0603151 service placement] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:01.719146 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97d082-401d-40ba-84ce-8e5cd0603151 service placement] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:01.768873 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea72321-9612-4ba8-a730-4f5fb26cd6ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de05c08e30dd4fff93f1c97aac9553d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:01.769531 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea72321-9612-4ba8-a730-4f5fb26cd6ca service neutron] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:01.769865 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea72321-9612-4ba8-a730-4f5fb26cd6ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:01.770983 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea72321-9612-4ba8-a730-4f5fb26cd6ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:01.771871 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-5ea72321-9612-4ba8-a730-4f5fb26cd6ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:01.834466 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea72321-9612-4ba8-a730-4f5fb26cd6ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:01.836505 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea72321-9612-4ba8-a730-4f5fb26cd6ca service neutron] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:01.846069 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1475/2957] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:02.025554 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:02.027327 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:02.051800 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da188b6d-0e09-4d40-817c-f566b200d34d admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:02.067190 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:02.067970 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:02.068355 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:02.068731 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] PATH_INFO: `/v3/projects/e4fd5ac76e47431eb9ed60b34784d2e3` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:02.069995 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4fd5ac76e47431eb9ed60b34784d2e3)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:02.072433 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:02.128006 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4fd5ac76e47431eb9ed60b34784d2e3', 'request_id': 'req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.134754 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4fd5ac76e47431eb9ed60b34784d2e3', 'request_id': 'req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.139732 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:02.141011 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4fd5ac76e47431eb9ed60b34784d2e3', 'request_id': 'req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.156520 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] The token cache is being invalidate because project e4fd5ac76e47431eb9ed60b34784d2e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:40:02.156693 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.157948 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.160007 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1483/2958] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:40:02 2022] DELETE /identity/v3/projects/e4fd5ac76e47431eb9ed60b34784d2e3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:02.163906 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea72321-9612-4ba8-a730-4f5fb26cd6ca service neutron] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:02.173878 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea72321-9612-4ba8-a730-4f5fb26cd6ca service neutron] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:02.232025 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:02.232676 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:02.232954 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:02.233225 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] PATH_INFO: `/v3/users/63972a1995e843d4a077a19ef92146bc` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:02.234418 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63972a1995e843d4a077a19ef92146bc)` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:02.236405 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:02.277336 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:02.284387 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63972a1995e843d4a077a19ef92146bc'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.291768 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63972a1995e843d4a077a19ef92146bc'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.292467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63972a1995e843d4a077a19ef92146bc'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.293201 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63972a1995e843d4a077a19ef92146bc'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.305355 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.notifications [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63972a1995e843d4a077a19ef92146bc'} {{(pid=81795) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.317945 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1476/2959] 10.222.0.61 () {58 vars in 1224 bytes} [Wed Sep 7 13:40:02 2022] DELETE /identity/v3/users/63972a1995e843d4a077a19ef92146bc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:40:02.488354 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] Authenticating user token {{(pid=81796) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:02.489829 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.fernet_utils [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81796) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:02.508389 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209ac7c-6100-4e56-b1e3-b95077ab3a3d admin admin] Validating token access rules against request {{(pid=81796) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:02.570585 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2cf4bf-8469-4869-be27-2e20f251c164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bfb2d4384094e9e8a8b910061e5c9d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f256a069ba947c6a6b8cccfea6c7a22', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81796) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:02.571857 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cf4bf-8469-4869-be27-2e20f251c164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:02.572119 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cf4bf-8469-4869-be27-2e20f251c164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:02.572395 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cf4bf-8469-4869-be27-2e20f251c164 admin admin] PATH_INFO: `/v3/projects/13833edf7e68484390c47d0f7c797bad` {{(pid=81796) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:02.573546 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2cf4bf-8469-4869-be27-2e20f251c164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13833edf7e68484390c47d0f7c797bad)` {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:02.575485 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2cf4bf-8469-4869-be27-2e20f251c164 admin admin] RBAC: Authorization granted {{(pid=81796) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:02.624259 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4f2cf4bf-8469-4869-be27-2e20f251c164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13833edf7e68484390c47d0f7c797bad', 'request_id': 'req-4f2cf4bf-8469-4869-be27-2e20f251c164'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.630949 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4f2cf4bf-8469-4869-be27-2e20f251c164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13833edf7e68484390c47d0f7c797bad', 'request_id': 'req-4f2cf4bf-8469-4869-be27-2e20f251c164'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.637808 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4f2cf4bf-8469-4869-be27-2e20f251c164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13833edf7e68484390c47d0f7c797bad', 'request_id': 'req-4f2cf4bf-8469-4869-be27-2e20f251c164'} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.650960 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4f2cf4bf-8469-4869-be27-2e20f251c164 admin admin] The token cache is being invalidate because project 13833edf7e68484390c47d0f7c797bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81796) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:40:02.651266 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4f2cf4bf-8469-4869-be27-2e20f251c164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.651902 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG keystone.notifications [None req-4f2cf4bf-8469-4869-be27-2e20f251c164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81796) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.653396 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: [pid: 81796|app: 0|req: 1484/2960] 10.222.0.61 () {58 vars in 1233 bytes} [Wed Sep 7 13:40:02 2022] DELETE /identity/v3/projects/13833edf7e68484390c47d0f7c797bad => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:12.314395 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:DELETE=5,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:12.647606 dv-cinder-804035-50-iscsi devstack@keystone.service[81796]: DEBUG dbcounter [-] [81796] Writing DB stats keystone:SELECT=54,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81796) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:35.125819 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] Authenticating user token {{(pid=81795) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:35.127263 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:35.200156 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe79e8-3ad5-422a-b51c-9b7cfe58f837 admin admin] Validating token access rules against request {{(pid=81795) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:35.260121 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109ce21-6c66-4c92-9eab-366d78795ca9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e283e62741f40439d02e545a212807a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1ee05b1d00e4f4bbb39b9cecdfe754d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81795) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:35.260734 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109ce21-6c66-4c92-9eab-366d78795ca9 service placement] REQUEST_METHOD: `GET` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:35.261015 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109ce21-6c66-4c92-9eab-366d78795ca9 service placement] SCRIPT_NAME: `/identity` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:35.261805 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109ce21-6c66-4c92-9eab-366d78795ca9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81795) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:35.263590 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.fernet_utils [None req-1109ce21-6c66-4c92-9eab-366d78795ca9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81795) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:35.275178 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109ce21-6c66-4c92-9eab-366d78795ca9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:35.276764 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109ce21-6c66-4c92-9eab-366d78795ca9 service placement] RBAC: Authorization granted {{(pid=81795) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:35.287467 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: [pid: 81795|app: 0|req: 1477/2961] 10.222.0.61 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:45.282949 dv-cinder-804035-50-iscsi devstack@keystone.service[81795]: DEBUG dbcounter [-] [81795] Writing DB stats keystone:SELECT=58 {{(pid=81795) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}